software defined storage @ microsoft · vmware servers to azure storage in minutes with no...
TRANSCRIPT
PRESENTATION TITLE GOES HERE
Software Defined Storage @ Microsoft
Siddhartha Roy Cloud + Enterprise Division
Microsoft Corporation
Lessons Learned – operating large cloud properties
Industry trends
3
Virtualization • Mobility and flexibility within and
across sites/clouds • More density
Cloud scale services influencing design • Standard, volume hardware to build
high-scale systems at low cost • Cloud design points in software
Data explosion • Device proliferation • Modern apps • Unstructured data analytics
Scale Out with minimal operational complexity
What is Software Defined Storage (SDS) ? Cloud scale storage and cost economics
on standard, volume hardware
Microsoft Software Defined Storage (SDS) Breadth offering and unique opportunity
Customer choice
SAN and NAS storage
Private Cloud with partner storage
Windows SMB3 Scale Out File Server (SoFS) + Storage Spaces
Private Cloud with Microsoft SDS
StorSimple + Microsoft Azure Storage
Hybrid Cloud Storage
Microsoft Azure storage
Public Cloud Storage
Private Enterprise and Hosted Clouds with SAN and NAS storage
SAN and NAS storage
Private Cloud with partner storage
Windows SMB3 SoFS + Storage Spaces
Private Cloud with Microsoft SDS
StorSimple + Microsoft Azure
Hybrid Cloud Storage
Microsoft Azure storage
Public Cloud Storage
Optimizing Windows for SAN/NAS storage
• Technical work: SMI-S, NVM Programming, XAM
• SMB 2/3 Plugfest
• INCITS: T10,T11,T13 (SCSI,FC,AT) • IETF: Co chairs for NFS 4.1, STORM
(iSCSI)
Private Enterprise and Hosted Clouds with Microsoft Storage Defined Storage
SAN and NAS storage
Private Cloud with partner storage
Windows SMB3 SoFS + Storage Spaces
Private Cloud with Microsoft SDS
StorSimple + Microsoft Azure
Hybrid Cloud Storage
Microsoft Azure storage
Public Cloud Storage
Public Cloud lessons applied
Software Defined Storage in Windows Server 2012 R2
Primary application data storage on cost affective, continuously available, high performance
SMB3 file Shares backed by Storage Spaces
1. Performance, Scale: SMB3 File Storage network
2. Continuous Availability and Seamless Scale Out with File Server Nodes
3. Elastic, Reliable, Optimized Storage Spaces
4. Standard volume hardware for low cost
Storage Spaces
Hyper-V Clusters
SMB3 Storage Network Fabric
Shared JBOD Storage
1
4
2 2
3
Scale-Out File Server Clusters
SMB3 File Storage network High speed, low latency with SMB Direct SMB Multichannel for storage path failure resiliency Scale Out File Server Nodes Easy share management Active/Active Scale Out pay as you grow Continuously Available
Physical storage
SoFS + Storage Spaces – under the hood
Virtualized storage
Scale Out File Server cluster nodes
Hyper-V cluster nodes
SMB3 Storage Network
SSD
HDD
Tiered Storage spaces
Optimized Placement
Hyper-V Cluster
SMB Buffer
Scale Out File server (SoFS) Cluster
SoFS - Low latency, high scale with SMBDirect
App Buffer
SMB Buffer
Adapter Buffer
Adapter Buffer
• SMBDirect – Microsoft implementation of RDMA on Ethernet, InfiniBand over SMB3 protocol
• High throughput with low latency
• Optimized to expose full potential of 10/40Gbps Ethernet and 56Gbps InfiniBand
• Usage beyond storage for Server Server Fast Live migration
32KB random reads
from a mirrored space (disk)
~500,000 IOPS ~16.5 GBytes/sec
Software Defined Storage stack overview System Center Windows
Server
SMI-S Storage Service
Block storage provisioning
File storage provisioning
Hyper-V Storage
Management
Storage Pool Classification
Thin Provision
Alert Monitor
SAN based Rapid
Provisioning
SM API Integration
Storage Utilization Trending
PowerShell Support
SAS Array Support
Thin LUN provisioning
Cluster-Aware Updating with no
downtime
Guest Clusters on
SMB
8,000 VMs per Hyper-V Cluster
VM Prioritization
Scale with 64-node clusters
VM Storage Migration
Dynamic Access control
SMB3 in flight data encryption
Hyper-V Replica DPM
Windows Azure
Backup
SMB3 Direct for low latency
SMB3 SoFS for Hyper-V, SQL Server
SMB3 Multichannel for FT, Perf
SMB3 for Windows
clients
Dedup
SMB3 Transparent
Failover
iSCSI Target block
storage
ReFS local file system
for resiliency
NFS 3, 4.1 for *nix clients,
VMware
CSV Scale Out
Tiered Storage Spaces
Fast NTFS chkdsk
Private Cloud Choice - SAN or Microsoft SDS
Hyper-V compute nodes
Hyper-V compute nodes
Block protocol fabric Low latency network Management of LUNs Storage tiering Data deduplication RAID resiliency groups Pooling of disks High availability Persistent write-back cache. Copy offload Snapshots
Traditional FC/iSCSI SAN storage arrays
Microsoft SDS Storage (in Windows Server)
File protocol fabric Low latency with SMB3Direct Management of Shares Storage tiering (new with R2) Data deduplication Flexible resiliency options Pooling of disks. Continuous availability Persistent write-back cache.
(new with R2) SMB copy offload Snapshots
FC/SAS disk shelf
FC/SAS disk shelf
SAN/NAS
Shared SAS JBOD
Shared SAS JBOD
Scale Out File Server + Storage Spaces
Customer Quote: NTTX
“By using Storage Spaces, Hyper-V over SMB, and other new features, I eliminate big, lumpy capital expenditures.… I can buy inexpensive storage…and scale.” - Philip Moss, Chief Technology Officer, NTTX Select
NTTX designed a new datacenter around Windows Server 2012 in order to get new services to market faster and at a lower cost.
Benefits: Achieved greater flexibility for meeting customer needs. Got new offerings to market faster. Increased annual revenue opportunities to U.S.$18.6 million. Saved $5.8 million a year.
Hosting provider uses industry-standard storage (Microsoft SoFS + Storage Spaces) to slash storage costs by 30 %
http://www.raidinc.com/ http://www.dataonstorage.com/ http://www.fujitsu.com/fts/
http://www.quantaqct.com/ http://www.supermicro.com/ http://www.dell.com/
Participants in the 2013 edition of the SNIA SMB2 / SMB3 Plugfest
Santa Clara, CA – September 2013
Ecosystem of SMB server/client implementations
SMB PlugFest Occurs every year side-by-side with the
Storage Developer Conference (SNIA SDC) in September
Interaction across operating systems and SMB implementations
Hybrid Cloud Storage StorSimple + Azure
SAN and NAS storage
Private Cloud with partner storage
Windows SMB3 SoFS + Storage Spaces
Private Cloud with Microsoft SDS
StorSimple + Microsoft Azure
Hybrid Cloud Storage
Microsoft Azure storage
Public Cloud Storage
MICROSOFT – NDA ONLY
StorSimple Applications & Workloads Ca
pacit
y
Time
Data grows exponentially (50–60% Annually)
Most I/O happens to working set data
CapEx, OpEx of traditional
storage
Cloud Storage
Local Storage
Target Use Cases File Share
SMB NFS
File servers NAS
SharePoint
Business intelligence Collaboration Content & records management
Archives EMR/PACS Legal Construction
Media Engineering Logs, records
VMs Regional office storage
VM sprawl VM archives
StorSimple connects Windows, Hyper-V and VMware servers to Azure Storage in minutes with no application modification
• Consolidates primary, archive, backup, DR thru seamless integration with Azure
• Cloud Snapshots = revolutionary speed, simplicity and reliability for backup and recovery
• Reduces enterprise storage TCO by 60–80%
Application Servers
Inactive Primary Data + Backup + Archive + DR
Speed of SSD/SAS + Elasticity of Cloud
Warm data on SAS Local Tier
Most Active Data on SSD
StorSimple Appliance
Azure Storage
Public Cloud Storage – Azure
SAN and NAS storage
Private Cloud with partner storage
Windows SMB3 SoFS + Storage Spaces
Private Cloud with Microsoft SDS
StorSimple + Microsoft Azure
Hybrid Cloud Storage
Microsoft Azure storage
Public Cloud Storage
Store, backup, recover your data • Highly scalable: 20 trillion objects stored • Highly durable: Multiple copies of your data • Multiple object support: blobs, tables, drives • Easy access: Supports REST APIs • Available anywhere: multiple regions
Get your data wherever and whenever you need it • Microsoft Azure Storage • Defend against regional
disasters.
Geo replication
Continue the journey ……
Microsoft Cloud Server – Contribution to Open Compute Contributions to Open Compute Project – CAD models, Specs, Gerber files, Source code Benefits to traditional servers - 40% cost savings, 15% power efficiency gains
Where can I find more information ?
Microsoft Storage http://Microsoft.com/storage
Solution guide – Cost effective storage for Hyper-V http://technet.microsoft.com/en-us/library/dn554251.aspx
Drop by our expo booth