1 c jimmy may, mcm sql server on flash: re-thinking best practices sql server solutions architect |...
TRANSCRIPT
1c
Jimmy May, MCMSQL Server on Flash: Re-Thinking Best Practices
SQL Server Solutions Architect | Fusion ioMemory™ Solutions
1
DBA Fundamentals :: Down Under
2
Preamble
2
3
LegaleseDuring our meeting today, we may make forward-looking statements.
Any statement that refers to expectations, projections or other characterizations of future events or circumstances is a forward-looking statement, including those relating to market position, market growth, product sales, industry trends, supply chain, future memory technology, production capacity, production costs, technology transitions, construction schedules, production starts, and future products. This presentation contains information from third parties, which reflect their projections as of the date of issuance. Actual results may differ materially from those expressed in these forward-looking statements due to factors detailed under the caption “Risk Factors” and elsewhere in the documents we file from time to time with the SEC, including our annual and quarterly reports. We undertake no obligation to update these forward-looking statements, which speak only as of the date hereof.
©2014 SanDisk Corporation. All rights reserved. SanDisk is a trademark of SanDisk Corporation. Lightning, Lightning Eco, Lightning Ascend, Lightning Ultra, Optimus, Optimus Eco, Optimus Ascend, Optimus Ultra, Optimus Extreme, CloudSpeed, CloudSpeed Eco, CloudSpeed Ascend, CloudSpeed Ultra, CloudSpeed Extreme, FlashSoft, ZetaScale, Guardian Technology, FlashGuard, DataGuard, EverGuard, SanDisk ION Accelerator, Fusion ioMemory, Fusion ioSphere, Fusion ioTurbine, Fusion ioVDI, and others are trademarks of SanDisk Enterprise IP LLC. Other brand names mentioned herein are for identification purposes only and may be the trademarks of their respective holder(s).
4
Introduction
Flash storage is a game-changer for virtually every app in which it is used. Whether used locally—SATA, SAS, or PCIe; or for tiered storage or all-flash arrays—flash is changing what we thought we knew about SQL Server performance.
This presentation provides a brief survey of history and a high-level, vendor-agnostic review of options for implementing flash in the enterprise.
4
5
Jimmy May, MCM• SanDisk SQL Server Solutions Architect
Formerly:• Sr. Program Manager, SQL CAT• SQL Server Customer Advisory Team• MSIT Principal Architect: Database
Microsoft Certified Master: SQL Server (2009)MS IT Gold Star Recipient (2008) Microsoft Oracle Center of Excellence (2008)SQL Server MVP Nominee (2006)Indiana Windows User Group www.iwug.net
• Founder & Board of DirectorsIndianapolis Professional Association for SQL Server www.indypass.org
• Founder & Member of Executive CommitteeSQL Server Pros: Founder & Visionary-in-ChiefSQL Innovator Award Recipient (2006)
• Contest sponsored in part by MicrosoftFormerly Chief Database Architect for high-throughput OLTP VLDB at ExactTarget (recent IPO)Senior Database Administrator for OpenGlobe/Escientwww.twitter.com/aspiringgeek [email protected] [email protected] http://sqlblog.com/blogs/jimmy_may
Bio
6
Data Propulsion Laboratory — DPL
7
C:\>sqlservr.exe /faster
8
Acknowledgements Kevin Kline, MVP Brent Ozar, MCM, MVP Thomas LaRock, MCM, MVP Denny Cherry, MCM, MVP Gavin Payne, MCA, MCM Chuck Suter, Pure Storage Mike Anderson, Disney Mike Ruthruff, Bungie Studios Thomas Kejser Tim Radney, MVP Mindy Curnutt, MVP
Brian Walters, SanDisk Niall MacLeod, SanDisk John Gatch, SanDisk Peter Plamondon, SanDisk David Reeves, SanDisk Grant Jacobson, SanDisk Sean Hehir, SanDisk
9
Navigating this deck1.On the left, right-click on any section name
2.Click “Collapse All”)
3.Review the section names, then click to expand desired content
10
PASS Summit 2015 Pre-ConThe Complete Primer to SQL Server Virtualization
Learn More: http://www.sqlpass.org/summit/2015/Sessions/Details.aspx?sid=8149
11
SanDisk Speaking Sessions Title: The Complete Primer to SQL Server Virtualization (Full Day Session)
– Speakers: • Jimmy May, MCM, SQL Server Solutions Architect, SanDisk• David Klee, MVP, Founder & Chief, Heraflux Technologies• Argenis Fernandez, MVP, MCM, SQL Server Solutions Architect, Pure Storage
– Date: Monday, October 26– Time: 9 a.m. – 6:00 p.m. PST– Location: 611 – 614– Learn More: www.sqlpass.org/summit/2015/Sessions/Details.aspx?sid=8149 [DBA-299-P]
Title: SQL Server 2016 on Flash– Speaker: Brian Walters, Solution Architect, SanDisk– Date: Wednesday, October 28– Time: 10:15 a.m. PDT– Location: TCC101
Title: Columnstore Indexes in SQL Server: Flipping the DW Faster Bit [AD-206]– Speaker: Jimmy May, MCM, SQL Server Solutions Architect, SanDisk– Date: Wednesday, October 28– Time: 4:45 p.m. – 6:00 p.m. PST– Location: 6A– Learn More: www.sqlpass.org/summit/2015/Sessions/Details.aspx?sid=8623
Title: SQL Server on Flash: Re-Thinking Best Practices [DBA-210]– Speaker: Jimmy May, MCM, SQL Server Solutions Architect, SanDisk and Brian Walters, Solution Architect, SanDisk– Date: Friday, October 30– Time: 8:00 – 9:15 a.m. PST– Location: TCC 304– Learn More: www.sqlpass.org/summit/2015/Sessions/Details.aspx?sid=8601
SanDisk Theater– Visit the SanDisk booth (#22) for brief sessions—throughout the day, throughout the week—for an opportunity to learn & to win!
12
Agenda
12
13
Agenda History and Introduction to Enterprise Flash Flash Best Practices for SQL Server SQL Server Use Cases and Implementation Details Flash Myths, Misinformation,
& Re-visiting Best Practices PCIe Application Accelerators Microsoft SQL Server 2014 Fast Track (DWFTRA) Hardware for Nothing, Get Your Flash for Free
14
Enterprise Flash Unlocks Microsoft SQL Server 2014 Business Value
• Increase business productivity – more transactions, faster
• Improve customer experience – faster response times
• Reduce costs – scale more users and workloads per server
• Increase SLAs – faster SQL database startup, failover, & maintenance
15
Flash: Don’t be this guy. Don’t be in this org.
Hat tip to Bob Pusateri (@SQLBob)
via Jason Horner (@jasonhorner) & Argenis Fernandez (@DBArgenis )
16
Introduction to Enterprise Flash
16
17
Flash is Revolutionizing Enterprise Storage
19
May’s I/O Mantra
X capacity (GB)at Y throughput (MB/sec)at Z IOPs (transfers/sec) at <=10ms latency for OLTP data files
at <=30ms latency for DW data files at 0ms-2ms for log files
20
What is I/O?Performance Monitor: Logical Disk and PhysicalDisk Latency
– Measured in milliseconds (ms) – Measured in microseconds (µs)– Performance Monitor: Logical Disk
• Avg. disk sec/transfer• Avg. disk sec/read • Avg. disk sec/write
IOPS• Disk transfers/sec• Disk read/sec• Disk writes/sec
Throughput• Disk bytes/sec• Disk read bytes/sec• Disk write bytes/sec
Capacity– Measured in GB/TB– The easy one!
21
What is the business impact of flash on I/O?Google found an extra half-second in search-page generation time dropped traffic by 20%.
Amazon found every 100ms of latency cost them 1% in sales.
If a trading platform is 5 milliseconds behind the competition, it could lose millions in revenues per millisecond.
Performance is paramount: Asking users to wait is like asking them to leave.
If it is fast and ugly, they will use it and curse you; if it is slow, they will not use it. —Computer science professor, entrepreneur, & billionaire David CheritonApplication performance is not an event or something you do once—it's a practice. —Mark Kremer, CEO, Precise Software Solutions Inc.
22
Spinning Media vs. PCIe
How many conventional hard drives to match the IOPs of a single PCIe flash device?
23
24
Spinning Media vs. PCIe: Some Math Note: We could do lots of calculations with various types of I/O
– 4KB IOPs are one of the most commonly published– Numbers will vary for I/O characteristic of SQL Server – The ratios are nonetheless equally remarkable
Random per 4K Read IOPs– 600GB 15K HDD: 181– 3.2TB PCIe Flash: 345,000
Devices for same Random Read IOPs – 340,000 IOPs/PCIe device ÷ 181 IOPs/15K HDD = 1,906 HDDs/PCIe device
Height– 1,906 * 1” height/HDD * 1 foot/12” * 1 story/10 foot = 15.8 stories
Weight– 1,906 HDDs/PCIe * 1.51 pound/HDD = 2,878 pounds
Sources– https://www.sandisk.com/business/datacenter/products/flash-devices/pcie-flash/sx350– http://www.seagate.com/files/docs/pdf/datasheet/disc/cheetah-15k.7-ds1677.3-1007us.pdf– http://archive.benchmarkreviews.com/index.php?option=com_content&task=view&id=375&Itemid=60&limitstart=5
25
Relative Performance of Conventional HDD vs. Flash There are many ways to look at disk performance:
– Read/Write– Random/Sequential– IOPs/Throughput– Block size impacts IOPs, throughput, & latency– Let’s look briefly at one perspective: IOPs vs.
latency
Spinning media (15K Enterprise SAS)– Random 4K reads: 210 IOPs @2000 µs– Random 4K writes: 190 IOPs @2000 µs
Flash vendor 1:– Random 4K reads: 75,000 IOPs @50µs– Random 4K writes: 36,000 IOPs @65µs
Flash vendor 2: – Random 4K reads: 252,000 IOPs @29µs– Random 4K writes: 236,000 IOPs @29µs
Spinning Media vs. Flash
180x – 1,000x+ IOPs
30x – 70x faster
38
Flash Best Practices for SQL Server
38
39
Flash Best Practices for SQL Server
Though flash can seem like a magic bullet …– Flash can mask many performance, configuration, and design challenges
Best practices remain fundamental There is no substitute for engineering discipline in terms of
application & database design
If you don't have time to do it right, when will you have time to do it over?
—John Wooden
40
Workload Characteristics and Pain Points
OLTP Small block, random I/O High IOPS Low throughput Low Latency
Data Warehouse Large block, sequential I/O Lower IOPS High throughput High Latency
41
Flash Addresses Pain Points
Pain Points Overprovisioned Shared
Storage – random I/O Introduces I/O Bottleneck:
– High Latency– Low IOPS – Low CPU utilization– High Cost / Depressed Return
Flash Liberation Solid State minimizes seek time
– Ultra Low latency (µs)– Small & Large Block Size– High Throughput (29 GB/s @ 2 ms)– Low cost / High return
42
Flash Best Practices: SQL Server Configuration tempdb Configuration
– -T1117 (to enable consistent AutoGrow)– -T1118 (to eliminate SGAM contention)– Multiple, equisized tempdb data files– Note: These are baked into SQL Server 2016– Consider making these mandatory components for all SQL Server deployments
Local Security Policy – secpol.msc > Local Policies > User Rights Assignment– Add SQL Server service account to the following:
• Perform volume maintenance tasks (to enable Instant File Initialization).
• Lock pages in memory (to prevent paging).
43
Best Practice – SQL Server
Mitigate memory latch contention:• User DBs: Multiple equally-sized data files per data file filegroup• Tempdb: Multiple equally-sized data files • 8 files is the sweet spot for most implementations
Consider segregating log files from user and tempdb data files if demanded by performance
Configure 'optimize for ad hoc workloads' – mitigate single use statements from overly consuming plan cache & from monopolizing excessive CPU
Dispelling some myths...
44
Flash Best Practices: Windows Power Configuration Power options
– Windows power options:Control Panel > Hardware > Power Options >
High Performance
PoSh: Powercfg -SETACTIVE SCHEME_MIN
45
Flash Best Practices: BIOS
Logical processor / hyper-threading: Enable– Contemporary hyper-threading vs. the original incarnation
Fan: high or increased cooling C-States: Disable all “power saving” options
– Configure for maximum clock speed & voltage– Note: Sometimes called P-States
46
Flash Best Practices: Storage Devices Logical volume configuration
– Disk Partition Alignment remains important– Windows-level format (FAUS): 64KB
Drivers: – Update drivers– More important for flash devices than for conventional spinning media
Vendor-specific recommendations:– Default power settings for device may not be optimal
• This is in addition to physical server & Windows power settings
– “Low-level” formatting: 4KB (not 512B) recommended
47
Power Optimization Summary Maximize power settings in three different locations:
– BIOS• Disable C-State settings
• Maximize fan or cooling, if available
– Windows: High Performance– PCIe configuration
48
Flash Best Practices: Bandwidth Maximum performance is limited by system bottlenecks 2.5” flash drives in server front-end
– It doesn’t take much flash to overwhelm last gen buses– Contemporary buses can accommodate 20 – 24 disks
• Up to 100TB!
• Lenovo X3650 w/Optimus Max 100TB storage (100sec)www.twitter.com/aspiringgeek/status/615909659916898304
– Be aware of potential thermal issues
Validate PCIe slots best-suited for cards– PCIe version– Bus speed: 4x, 8x, 16x– Bus power
49
Best Practice – SQL Server Backup
Design a User & System Database Backup strategy compliant with SLOs & SLAs
Backup Types: FULL, DIFFERENTIAL, LOG, COPY_ONLY Test & Validate:
– Full, Diff, Log—including point-in-time– Integrity: DBCC CHECKDB
You don’t have a backup until it’s been restored—& validated. —Kimberly Tripp
50
Best Practice – SQL Server Maintenance Fragmentation
– SQL Server “read ahead” relies on logically contiguous pages
SORT_IN_TEMPDB Index Build Option– Significant reduction in I/O within SQL Server data files– Especially important for AlwaysOn
51
Backup & Restore Performance Options Backup
– Backup File Count (multiple streams)– COMPRESSION– MAXTRANSFERSIZE
• 4,194,304 (4MB) is max permitted block size– BUFFERCOUNT
• 2x to 4x logical core count Restore
– Presumably multiple files & compression were implemented during backup– Leverage MAXTRANSFERSIZE & BUFFERCOUNT
Note: Consider enabling these options via sp_configure– backup checksum default– backup compression default
52
Prototypical T-SQL BACKUP SyntaxDECLARE @bkpname sysname = 'MyFastBackup‘ -- NAME is exposed via RESTORE HEADERONLY FROM DISK = 'path to first bkp file‘ -- & also in SELECT * FROM msdb.dbo.backupsetBACKUP DATABASE MyDB TO DISK = '<local or \\UNC path>\DBbkps\MyDB_20150909_#1.bak' , DISK = '<local or \\UNC path>\DBbkps\MyDB_20150909_#2.bak' , DISK = '<local or \\UNC path>\DBbkps\MyDB_20150909_#3.bak' , DISK = '<local or \\UNC path>\DBbkps\MyDB_20150909_#4.bkp' WITH NAME = @bkpname , STATS = 1 , COMPRESSION , MAXTRANSFERSIZE = 4194304 -- 4MB is max permitted block size , BUFFERCOUNT = 96 -- 2x to 4x logical core count , STOP_ON_ERROR -- CHECKSUM error
53
Prototypical T-SQL RESTORE SyntaxDECLARE @bkpname sysname = 'MyFastBackup‘ -- NAME is exposed via RESTORE HEADERONLY FROM DISK = 'path to first bkp file‘ -- & also in SELECT * FROM msdb.dbo.backupsetRESTORE DATABASE MyDB FROM DISK = '<local or \\UNC path>\DBbkps\MyDB_20150909_#1.bkp' , DISK = '<local or \\UNC path>\DBbkps\MyDB_20150909_#2.bkp' , DISK = '<local or \\UNC path>\DBbkps\MyDB_20150909_#3.bkp' , DISK = '<local or \\UNC path>\DBbkps\MyDB_20150909_#4.bkp' WITH REPLACE , RECOVERY , STATS = 1 , MAXTRANSFERSIZE = 4194304 --4MB is the max allowed , BUFFERCOUNT = 112 --up to 4xcorecount , STOP_ON_ERROR -- CHECKSUM error , MOVE 'tpch_300_Primary_SystemData01' TO 'C:\mount\data\MyDB_Primary_SystemData01.mdf' , MOVE 'tpch_300_Primary_SystemData02' TO 'C:\mount\data\MyDB_Primary_SystemData02.mdf' , MOVE 'tpch_300_Primary_SystemData03' TO 'C:\mount\data\MyDB_Primary_SystemData03.mdf' , MOVE 'tpch_300_Primary_SystemData04' TO 'C:\mount\data\MyDB_Primary_SystemData04.mdf' , MOVE 'tpch_300_Secondary01' TO 'C:\mount\data\MyDB_Secondary01.ndf‘ , ...
, MOVE 'tpch_300_Log' TO 'C:\mount\data\MyDB_Log.ldf'
54
Automated Backup Tuning What’re the best options for your environment? Sir SQL—our knight in shining armor—with a PoSh-based solution!
Automated Backup Tuninghttps://bit.ly/SirSQLBackupTuning
Nic Cain @SirSQL
55
Data Compression Long a best practice for SQL Server data Yet it’s often not implemented Typically, relatively small CPU cost (3%) in exchange for storage
savings of ~2x – 5x Corollary: If extant data is not compressed, compression while
migrating to flash can effectively discount your new storage costs by half or more.
Note: Some flash vendors provide compression or de-dupe by default.
56
Comparing Space Savings
19.7GB
10.9GB
5.0GB4.0GB
6.9GB
1.8GB
56
57
#2014-003
#2014-003
#2014-002
#2014-009
#2014-023
#2014-001
#2014-028
#2014-032
0
500
1000
1500
2000
2500
3000
3500
4000
0
10
20
30
40
50
60
70
80
90
100
Query Thruput
RS MeasuredThruput(Queries/Hr/TB)
CS MeasuredThruput(Queries/Hr/TB)
Max User Capacity(TB)
(Que
ries/
Hr/T
B
TB
Row Store vs. Columnstore: I/O vs. Query Throughput
Copyright © 2015 SanDisk. All rights reserved.
#2014-003
#2014-003
#2014-002
#2014-009
#2014-023
#2014-001
#2014-028
#2014-032
0
1,000
2,000
3,000
4,000
5,000
6,000
7,000
8,000
9,000
10,000
0
10
20
30
40
50
60
70
80
90
100
I/O Thruput
RS Scan RatePhysical(MB/sec)
CS Scan RatePhysical(MB/sec)
Max User Capacity(TB)
MB/
sec
TB
58
Best Practice – Data Warehouses New deployments should consider Data Warehouse Fast Track
Reference Architectures (DWFTRA) Columnstore Indexes
– Best, underutilized feature in SQL Server 2014– Microsoft recommends that columnstore be the default choice for
warehouse tables– Leverage table partitioning
59
Stay Tuned for Guidance
-E & -T834– These options direct SQL Server to use Windows large-page
memory allocations for the buffer pool & large physical page allocations, respectively.
– We are investigating interesting memory and I/O behaviors when these traces flags are enabled with certain SQL Server features
Indirect Checkpoint Delayed Durability TDE (& Always Encrypted)
60
Flash Myths
60
61
Flash Myths, Misinformation, & Re-visiting Best Practices
Log files & data files Fragmentation OLTP vs. OLAP/DW Cost Durability/Reliability Availability & RAID for PCIe Cards CPU0 bottlenecks
62
Fragmentation Concept of “sequential” access has no physical analog in flash. Defragmentation remains a nominal best practice, especially for
OLTP/DW workloads – SQL Server readahead requires logically contiguous data
Yet even within Microsoft, benchmarks on flash are often run without the benefit of defragging(!)
63
Log Files & Data files
Once upon a time...– Physical segregation of indexes from data files was a best
practice• Storage abstraction rendered this largely unnecessary
– We still often needed to segregate log files from data files– However, this may no longer be necessary—test!
• Bottom line: Look for consistent, low (<1ms) log file latency
64
OLTP vs. OLAP/DW
Once upon a time...– We didn’t intermingle workloads– Today? Flash may render this moot– Depends on server resources & application workload
65
Availability & RAID for SAS, SATA, & PCIe Not all flash is created equal!
– Consumer vs. Enterprise
Contemporary Enterprise Flash is robust– Massively over-provisioned, internal error checking, etc.– Example warranty: 50PBW (petabytes written), i.e., 50 * 1024TB of writes
• 512GB each day for 6 years
• Doable, yet...
Is it necessary to RAID enterprise flash?– It depends– Be clear: I am not advocating an SPOF!– Perhaps a better use of resources to focus on server/system HA
66
CPU0 bottlenecks Not a myth, but antiquated Windows Software RAID leverages CPU0 Some flash drivers used to do so as well
– Result, high I/O could result in one core with little headroom– Even though other cores were relatively unused– Must examine core-by-core % Processor Time
For example: 16 core server, CPU0 at 99%, CPU1-15 at 10%– Average CPU <16%
Contemporary flash drivers render this moot because they either: – On-board device processor, or – Balance CPU across all cores
67
Cost Once upon a time...
– Flash was expensive– If you think flash is expensive, try pricing a SAN
• And supporting it...
– Yet time is money • Remember the latency slide?
– Today: • Glacial SATA is ~50¢/GB
• Depending on format, vendor, re-seller, etc., Enterprise Flash has recently been as economical as $3/GB & quickly approaching $1/GB
• As of March 2015: InfiniFlash is as low as $2/GB!
• SAS & SATA SSD offerings as low as 50¢ - $1/GB
68
Raw Cost/Capacity vs. Access Density
Raw Cost/Capacity – $/TB is as simplistic as classifying storage simply in terms of capacity– Remember the I/O Mantra: latency, IOPs, thruput, capacity
Consider: Access Density
AD = work ÷ capacity ÷ time• e.g., IOs (work) / GB (unit of storage) / second (unit of time)
Related: – Warehouse workloads rated in terms of Queries/Hr/TB
69
TCO & TCA: Flash is Fast—& Less Expensive than HDDs
The Accelerating Economics of Flash and the Retreat of Hard Disk Driveshttp://itblog.sandisk.com/the-accelerating-economics-of-flash-and-the-retreat-of-hard-disk-drives
15K SAS Enterprise HDD vs. 4TB Optimus Max
70
SQL Server Use Cases
70
71
SQL Server Use Cases: Single-Instance Options tempdb as a Cluster Shared Resource AlwaysOn HA using Readable Replicas AlwaysOn Multi-site HA with DR Windows Scale Out File Server (SOFS) Compression Fragmentation Flash for Development Environments
72
Entire Database on In-Server Flash
Parts of DB on In-Server Flash
(e.g., “hot files” or tempdb)
Cache data read from SAN on In-Server Flash
AlwaysOn Availability Groups ION Applianceas Shared Storage for FCI
FlashSoft®
Virtualized Database Servers for Consolidation, Upgrade, Mixed
Workloads
SQL Server 2014 Architectural Configurations
Data Warehouse Fast Track Reference Architectures (FTDWRA)
76
AlwaysOn Multi-site HA with DR
Synchronous
Asynchronous
Primary (server 1)
Local Replica
Remote Replica
Primary DC Disaster Recovery DC
Availability Group
77
Windows Scale Out File Server (SOFS) Software-defined Storage
https://en.wikipedia.org/wiki/Software-defined_storage– Software-defined storage (SDS) is evolving concept for computer data storage software to manage policy-based provisioning and management of
data storage independent of hardware. Software-defined storage definitions typically include a form of storage virtualization to separate the storage hardware from the software that manages the storage infrastructure. The software enabling a software-defined storage environment may also provide policy management for feature options such as deduplication, replication, thin provisioning, snapshots and backup.
– By consensus and early advocacy, SDS software is separate from the hardware it is managing.
Windows implementation of a virtual SAN (VSAN) aka SANless storage JBODs on steroids
– Storage Spaces– Storage Spaces Direct
Features– Redundancy– Performance– Amazingly cheap—er, ah inexpensive
Introduced in Windows Server 2012, significant enhancements in Windows Server 2016 Be aware—pay attention—we’ll be hearing more-&-more about Software-defined Storage & SOFS Windows Server 2012 Scale-Out File Server for SQL Server 2012 - Step-by-Step Installation by Jose Barreto
78
Windows Scale Out File Server (SOFS)
7979
SanDisk: Test ScenarioEnhanced AlwaysOn
Workload scenario High volume transactional workload I/O
Small block reads & writes (8K – 16K) Random reads & writes
Continuous writes to transaction log Synchronous availability mode High volume log transport across the
wire Likewise high volume for redo on
secondary
8080
Systems Under Test (SUT) 2 Lenovo ThinkServer RD650 Servers
each with:
SanDisk Fusion ioMemory
6.4TB
3.2TB
Processors
2 socket, 56 cores ~2.6Ghz Haswell-EP
Network
2x Mellanox ConnectX-3 Ethernet Adapter 56Gbs (7MB/s)
BIOS set to high performance
Fan offset increased
C-States disabled
Virtualization disabled
Load Injector (LI) 256GB DRAM 16 cores/32 LP Windows Server 2012 R2 10Gbe NIC
ID
RD650
ID
RD650
SUTs
LI
Load Drivers
SanDisk: Hardware Configuration
ID
RD650
Workload: OLTP HekatonHA: AlwaysOn
Enhanced AlwaysOn
8181
Windows Server 2012 R2 Datacenter
Windows High Performance(Powercfg -SETACTIVE SCHEME_MIN)
secpol.msc Lock pages in memory Perform volume maintenance tasks
SanDisk: OS & SQL Server ConfigurationSQL Server 2016 CTP2.2+SQL Server 2014 SP1
Server configuration options: max server memory: 128000 max worker threads: 2016 (default:
1344) AlwaysOn configuration
Availability mode: SYNCHRONOUS_COMMIT
Failover mode: AUTOMATIC Workload: TPCC Hekaton, assigned
to a Resource Governor resource pool: CREATE RESOURCE POOL
Pool_Hekaton WITH (MIN_MEMORY_PERCENT = 50, MAX_MEMORY_PERCENT = 90);
82
AlwaysOn Testing: SQL Server 2014 vs. 2016The engine & the hardware are liberated to do what they’re supposed to do Processor: ~4x CPU
More transactions: >5x XTP transactions
More flash: ~3x throughput; >4x IOPs; 0ms – 1ms latency
Source of ImprovementAlwaysOn Transport: >4x Log Bytes Received
83
AlwaysOn Log Synchronization
84
Application Performance
CPU (%) XTP Transactions/sec (x1000)
Log Bytes Received (MB/s)
Redone Bytes/sec (MB/s)
0
50
100
150
200
250
300
350
400
450
500
12 22
110 109
52
120
467
117
SQL2014 SP1
~4x CPU Utilisation
>5x XTP Transactions
>4x Log Bytes Received
85
SanDisk Flash Performance
Disk: IOPs (x1000) Disk: Thruput (MB/s) Disk: Latency (ms)0.000
100.000
200.000
300.000
400.000
500.000
600.000
700.000
800.000
900.000
1,000.000
18
200
053
873
1
SQL2014 SP1 SQL2016 CTP2.2
~3x IOPs
>4x Throughput
0ms – 1ms Latency
86
Performance PatternCorrelation: ProcessorAlwaysOn TransportXTP TransactionsFlash
87
Flash for Development Environments Consider using flash for dev environments I’ve always been a fan of providing developers with a robust
environment Providing a flash platform (& plenty of all the right resources)
encourages experimentation & innovation
88
Microsoft SQL Server 2014 Fast Track (DWFTRA)
88
89
Data Warehouse Fast Track Reference Architectures(DWFTRA)
Perfectly balanced hardware solutions (CPU, memory, storage) Official Microsoft Certification issued for each RA OEMs often provide BoMs for the solutions, significantly
simplifying ordering (e.g., Lenovo’s doc_id) Integral for our upcoming SQL Server 2005 EOS consolidation story Numerous reference architectures published publicly
90
DWFTRA (cont.)
OEM Server RackUnits
Protocol DB Size(TB)
MeasuredThruput
(Queries/Hr/TB)
Microsoft RACertification
Date OEMDocumentation
HP DL380 G8 2 PCIe 28 202 #2014-003* 9/3/2014 linkHP DL380 G8 2 PCIe 45 198 #2014-003* 9/3/2014 linkHP DL580 G8 4 PCIe 90 366 #2014-002 9/3/2014 link
Lenovo x3650 M5 2 PCIe 20 265 #2014-009 10/15/2014 linkLenovo x3850 X6 4 PCIe 55 360 #2014-023 3/10/2015 linkLenovo x3850 X6 4 PCIe 60 371 #2014-001 8/11/2014 linkLenovo x3850 X6 4 PCIe 95 433 #2014-028 5/5/2015 linkLenovo x3650 M5 2 SAS 37 192 #2014-032 7/14/2015 link
*#2014-003 reflect two certifications issued on the same day & available in the same document.
91
DWFTRA (cont.)
Internal resources– DPL: Sprocket >> MSSQL2014FTRA – EPS: Sprocket >> SQL Server > Reference Architectures and Tech Guides
Pros – These RAs are elegant solutions & the DPL strongly endorses them– DWFTRA are inexpensive & compact compared to spinning media – The Fast Track program is mature—Microsoft is currently on v.3, work on v.4 underway– The DPL collaborates closely with the Microsoft Fast Track team
Cons– Customer awareness is low– Microsoft incents sales for APS (formerly PDW) & Azure, so DWFTRA has a low profile
92
HP 28TB Data Warehouse Fast Track CertificationDWFT Certification
#2014-003 HP ProLiant DL380 Gen8 with HP PCIe LE Workload
Accelerator 28TB reference architecture for Microsoft SQL Server 2014 Data Warehouse Fast Track
Report Date 9/3/2014
DWFT Rev. 5.4
System Provider System Name Processor Type Memory
HP DL380 G8 Intel Xeon E5-2697v2 2.7 GHz (12/24/48)
768 GB
Operating System SQL Server Edition
Windows Server 2012 R2 SQL Server 2014 Enterprise Edition
Storage Provider Storage Information
4 x 2.6TB HP PCIe LE Workload Accelerator for data and tempdb 2 x 300GB HDD for OS (RAID 1)
8 x 15K 300GB HDD for log (RAID 10)
Primary Metrics
Rated User Data Capacity1
(TB) Row Store Relative
Throughput2 Column Store Relative
Throughput3 Maximum User Data Capacity
(TB)
28 170 218 32
Row Store
Relative Throughput2
Measured Throughput
(Queries/Hr/TB)
Measured Scan Rate Physical
(MB/Sec)
Measured Scan Rate Logical
(MB/Sec)
Measured I/O Throughput
(MB/Sec)
Measured CPU (Avg.)
(%)
170 202 4,508 5,500 5,004 96
Column Store
Relative Throughput2
Measured Throughput
(Queries/Hr/TB)
Measured Scan Rate Physical
(MB/Sec)
Measured Scan Rate Logical
(MB/Sec)
Measured I/O Throughput
(MB/Sec)
Measured CPU (Avg.)
(%)
218 1,414 1,389 N/A N/A 98
Breakthrough Performance - 5.5GB/s throughput - 1414 queries / Hr / TB - 98% average CPU utilization
SanDisk-based 2.6TB HP Workload Accelerators
HP DL380 Server
93
Breakthrough Performance – 11.4GB/s scan rate
Lenovo 60TB SQL 2014 Data Warehouse - Ref Architecture
System x 3850 X6 Server
SanDisk-based 2400GB Lenovo io3 Flash Adapters
94
HP 45TB Data Warehouse Fast Track CertificationDWFT Certification
#2014-003 HP ProLiant DL380 Gen8 with HP PCIe LE Workload
Accelerator 45TB reference architecture for Microsoft SQL Server 2014 Data Warehouse Fast Track
Report Date 9/3/2014
DWFT Rev. 5.4
System Provider System Name Processor Type Memory
HP DL380 G8 Intel Xeon E5-2697v2 2.7 GHz (12/24/48)
768 GB
Operating System SQL Server Edition
Windows Server 2012 R2 SQL Server 2014 Enterprise Edition
Storage Provider Storage Information
4 x 5.2TB HP PCIe LE Workload Accelerator for data and tempdb 2 x 300GB HDD for OS (RAID 1)
8 x 15K 300GB HDD for log (RAID 10)
Primary Metrics
Rated User Data Capacity1
(TB) Row Store Relative
Throughput2 Column Store Relative
Throughput3 Maximum User Data Capacity
(TB)
45 161 227 72
Row Store
Relative Throughput2
Measured Throughput
(Queries/Hr/TB)
Measured Scan Rate Physical
(MB/Sec)
Measured Scan Rate Logical
(MB/Sec)
Measured I/O Throughput
(MB/Sec)
Measured CPU (Avg.)
(%)
161 198 4,220 5,240 4,730 97
Column Store
Relative Throughput2
Measured Throughput
(Queries/Hr/TB)
Measured Scan Rate Physical
(MB/Sec)
Measured Scan Rate Logical
(MB/Sec)
Measured I/O Throughput
(MB/Sec)
Measured CPU (Avg.)
(%)
227 1,476 1,443 N/A N/A 99
Breakthrough Performance - 5.2GB/s throughput - 1476 queries / Hr / TB - 45TB compressed data in 2U
HP DL380 Server
SanDisk-based 5.2TB HP Workload Accelerators
95
HP 90TB Data Warehouse Fast Track CertificationDWFT Certification
#2014-002 HP ProLiant DL580 Gen8 with HP PCIe LE Workload
Accelerator 90TB reference architecture for Microsoft SQL Server 2014 Data Warehouse Fast Track
Report Date 9/3/2014
DWFT Rev. 5.4
System Provider System Name Processor Type Memory
HP DL580 G8 Intel Xeon E7-4890 v2
2.8 GHz (4/60/120) 1536 GB
Operating System SQL Server Edition
Windows Server 2012 R2 SQL Server 2014 Enterprise Edition
Storage Provider Storage Information
6 x 5.2TB HP Workload Accelerator PCIe Flash devices for data and tempdb 2 x 500GB 7200RPM HDD for OS (RAID 1)
2 x 1.3TB HP Workload Accelerator PCIe Flash devices for log (RAID 1)
Primary Metrics
Rated User Data Capacity1
(TB) Row Store Relative
Throughput2 Column Store Relative
Throughput3 Maximum User Data Capacity
(TB)
90 314 419 108
Row Store
Relative Throughput2
Measured Throughput
(Queries/Hr/TB)
Measured Scan Rate Physical
(MB/Sec)
Measured Scan Rate Logical
(MB/Sec)
Measured I/O Throughput
(MB/Sec)
Measured CPU (Avg.)
(%)
314 366 8,351 10,139 9,245 92
Column Store
Relative Throughput2
Measured Throughput
(Queries/Hr/TB)
Measured Scan Rate Physical
(MB/Sec)
Measured Scan Rate Logical
(MB/Sec)
Measured I/O Throughput
(MB/Sec)
Measured CPU (Avg.)
(%)
419 2,721 3,392 N/A N/A 96
Breakthrough Performance - 10.1GB/s throughput - 2721 queries / Hr / TB - 90TB compressed data in 4U
HP DL580 Server
SanDisk-based 5.2TB HP Workload Accelerators
96
Hardware for Nothing, Get Your Flash for Free
96
97
SQL Server Licensing Labyrinth
98
Licensing Variables
Cores & Sockets SQL Server Edition Virtualized vs. Bare Metal Host vs. Guest Software Assurance (SA) or not Active or Passive Legacy Licensing Is your sales rep under quota & is it near end-of-quarter or better yet near end-of-fiscal
year? Which highly-trained professional sales rep you talk to
– Ask 2 people, get 3 opinions
99
Weasel Clause Licensing is complex Directions on Microsoft has a multi-day boot camp costing
thousands of dollars dedicated to Microsoft licensing– Windows & Office portion: 4 – 6 hours– SQL licensing: several more hours
Failure to respect the complexities risks leaving out so many important details that we’ll be “wrong” in some aspects
Yet the case studies which follow are eminently defensible
100
Data Center Economics
SanDisk Flash isn’t transformative merely because of performance - the economics are disruptive
5 SQL Server Licensing/Hardware scenarios– #1. Using Flash to eliminate AlwaysOn Read-only Secondaries & their
licenses– #2. The Simple Upgrade: Spare compute & budget– #3. Application-level: Scaling up to big money– #4. Data Center Scale: Literally saving millions– #5. ...
Hardware for Nothing, Get Your Flash for Free
101
#1. AlwaysOn Readable Secondaries
3 Sustained Simultaneous Workstreams Scenario⁻ 2 HP DL580 4-socket servers, 64 logical cores per server⁻ Highly transactional TPCC-like environment – 1,000,000 New Orders per Minute⁻ 4 AlwaysOn Ags in synchronous availability mode, no queuing ⁻ Database backups at 4GB/s⁻ I/O Latency: < 0.0ms, i.e., µs range
Recommended Configuration⁻ Readable Secondary used for backup stream⁻ This requires licensing all cores on both servers
Configuration on SanDisk Fusion ioMemory Flash⁻ Backup offloaded from Secondary, so the Read-Only bit can be turned off⁻ This requires only one server to be licensed
SQL Server License Savings:⁻ 32 EE licenses (64 cores)
• @$13,473.00 USD = $ 431,136 (non-SA)• @$20,209.00 USD = $ 646,688 (SA)
103
Hardware for Nothing, Get Your Flash for Free
I want my...
I want my...
I want my SSDs!
104
#2. The Simple Upgrade Case Study
104
Scenario⁻ Four SQL Server instances⁻ Windows Server 2003 & SQL Server 2005 EOL approaching⁻ Four physical servers⁻ SAN attached
Solution₋ Four SQL Server instances₋ Windows Server 2012 R2 + SQL Server 2014 ₋ Three physical servers ₋ SanDisk PCIe Fusion ioMemory Flash ₋ ...with lots of compute & storage headroom
105
#2. The Simple Upgrade (cont.)
105
Consider:⁻ Dual socket quad-core to E5-2699 v3 octo-core, 128GB RAM @$9,943.50⁻ Each 2-core SQL Server EE license @$13,472.50
• 32 logical cores per server >> 16 SQL licenses per server >> $215,560.00⁻ Fusion ioMemory SX 6.4TB card @$39,779
Solution⁻ Classic hardware upgrade:
• 4 servers, each requiring 16 SQL licenses• 4 * ($9,944 + $215,560) = $ 902,016
⁻ SanDisk PCIe Application Accelerator solution:• 3 servers , again each requiring 16 SQL licenses + 3 Fusion ioMemory cards• 3 x ($9,944 + $215,560 + $39,779) = $ 795,849
⁻ Δ: Savings of $ 106,167⁻ …with lots of compute & storage headroom⁻ …enough to buy two more servers & Fusion ioMemory cards ⁻ And a leftover SAN!
106
Hardware for Nothing, Get Your Flash for Free
I want my...
I want my...
I want my SSDs!
THE CHALLENGE• Reduce power, cooling, and footprint• Near real time reporting for 15,000
users running up to 1.2 million queries per month
• Meet failover requirements
RESULTS• 3x more concurrent users• 3x more queries• 2x greater workload• Eliminated maintenance overhead for 27
disk arrays and over 400 disks• 16:1 footprint consolidation• 1/16th power and cooling
USAF Financial Data WarehouseTEKSOUTH
Source: Case Study - TekSouth Fights US Air Force Datacenter Sprawl with ioMemory; http://www.fusionio.com/case-studies/teksouth
#3. US Air Force & TEKSOUTH
108
#3. US Air Force System Changes (cont.)
109
#3. US Air Force System Changes Summary (cont.)
Before⁻ Database Servers (20RU)
⁻ 5 x 4U servers, four quad-core AMD Opteron™ 822 SE processors @ 3.0GHz, 32GB RAM⁻ 80 cores, 48 active
⁻ Storage⁻ Hard disks: 5 x 15K RPM SAS drives, 146GB Storage (81RU)⁻ 27 x 3U disk arrays⁻ Hard disks: 15 x 15K RPM SAS drives
After⁻ Database Servers (6RU)
⁻ 3 x 2U database servers, four hex-core Intel Xeon E5-4610 processors @ 2.40GHz, 64GB RAM⁻ 72 cores, 36 active
⁻ Storage⁻ Data stored on one ioDrive2 Duo 2.4TB and one ioDrive2 1.2TB in each server
Savings: (72-48) cores ÷ (2 cores/EE license) ÷ $13,472.50/license = $161,670
Case Study: http://www.fusionio.com/case-studies/teksouth
110
Hardware for Nothing, Get Your Flash for Free
I want my...
I want my...
I want my SSDs!
111
#4. Data Center Scale: Literally saving millions
Scenario– Windows Server 2003 & SQL Server 2005 EOL approaching– SAN-attached – Core SQL Server EE licenses @$13,472.50 each– 1,500 servers
Solution– Windows Server 2012 R2 leveraging Hyper-V– SQL Server 2014 leveraging AlwaysOn– Fusion ioMemory cards– 150 servers– Hundreds of fewer servers– Thousands of fewer licenses– Millions of dollars saved– ...with lots of compute & storage headroom– And leftover SANs & shelves & fiber
112
Hardware for Nothing, Get Your Flash for Free
I want my...
I want my...
I want my SSDs!
113
#5. <your company name here>
Scenario– Windows Server 2003 & SQL Server 2005 EOS approaching– New hardware likely to accompany software upgrades – SQL Server upgrades require per core licensing– 2-Core SQL Server EE license MSRP $13,472.50 (~$6,500/core)
Solution– Contemporary multi-core servers– Windows Server 2012 R2 leveraging Hyper-V– SQL Server 2014/2016 leveraging AlwaysOn, In-Memory, Columnstore, etc.– SanDisk Flash options:
• SanDisk Fusion ioMemory cards• SanDisk front-side SSDs• SanDisk FlashSoft caching software, SanDisk ION Accelerator™ shared storage,
etc.
114
Hardware for Nothing, Get Your Flash for Free
I want my...
I want my...
I want my SSDs!
115
Hardware for Nothing, Get Your Flash for Free
Challenges:– How to handle millions of IOPs/device, GBs of I/O throughput, µsec
latency– What to do with the leftover spinning media– What to do with the leftover budget?
Scenarios such as described here aren’t unusualThey’re typical, & they’re becoming more-&-more common
I reiterate: Actual customers today are literally saving hundreds of thousands of dollars per server
Takeaways
116
Upgrade for Nothing & the Flash is Free Customer after customer: Hundreds of thousands, even millions in
savings Licensing exercise / Consolidation story EOL: Windows Server 2003 & SQL Server 2005
– Upgrade h/w– Virtualize & Consolidate
117
SanDisk PCIe Application Accelerators
117
118
Accelerate your time-to-insight
Microsoft certified SQL 2014 solutions 9.2GB/s of I/O Throughput 700% improvement in query execution
times when using Clustered Columnstore Indexes
Database consolidation made easy. 3TB- 90TB size warehouses
Leveraging Fusion ioMemory Flash
119
Fusion ioMemory™ SX350Fusion ioMemory™ PX600
Leading the Industry: New! Improved! Gen 3.5
Capacities up to 6.4TBIndustry leading reliabilityStrongest ECC using LDPC
Unprecedented performance
NEW!
Fusion ioMemoryBlade Mezzanine Cards
Capacities up to 1.6TBFrom Cisco and HP
NEW!
120
Industry Leading Performance
Previous Fusion ioMemory™
Read Performance2X
HIGHEST
350K 4K Random Read IOPS
385K 4K Random Write IOPS
2.7GB/S ReadBandwidth
2.2GB/S WriteBandwidth
15μS WriteLatency
Write Performance in the Industry
121
Customer Testimonial: Plenty of Fish
At any given time, we can have over 700,000 users active, 30,000 messages a minute, 500,000 SQL statements per second across 30 servers.
The ROI is ridiculous. These cards pay for themselves in two weeks.
—Owen Morley Director of Infrastructure Plenty of Fish
https://youtu.be/zIWgmWEsvpM (2:13)
122
Thank You!