sqlserver 2008 fast track data warehouse
Post on 18-Nov-2014
121 Views
Preview:
DESCRIPTION
TRANSCRIPT
SQL Server 2008 Fast Track SQL Server 2008 Fast Track
Data Warehouse 2.0Data Warehouse 2.0
Speaker: Phil Hummel of WinWire Technologies
Presentation developed by: Bruce Campbell
Western Region Data Warehouse Specialist, Microsoft
Silicon Valley SQL Server User Group
February 16, 2009
Mark Ginnebaugh, User Group Leader,
mark@designmind.com
Agenda
• DW vs. OLTP
• Balanced Architecture Approach for DWFast Track Defined
• Fast Track Reference Architectures•
• Next Steps
END USER TOOLS
Microsoft DW & BI Stack
DELIVERY
PerformancePoint
Services
BI & DW PLATFORM (RDBMS, ETL, OLAP, Reporting)
DW versus OLTP
DatabaseDatabase•• Designed Designed for for analytical operations: operations:
Strategic Strategic focusfocus
•• Optimized Optimized for bulk load and large, complex, for bulk load and large, complex, unpredictable queriesunpredictable queries
•• Fewer concurrent users relative to Fewer concurrent users relative to OLTPOLTP
StorageStorage
DatabaseDatabase•• Designed Designed for for operational requirements: requirements:
Tactical Tactical focusfocus
•• Optimized for transactions: “single row” Optimized for transactions: “single row” entry and retrievalentry and retrieval
•• Thousands of concurrent usersThousands of concurrent users
StorageStorage
•• Primary focus on Read operationsPrimary focus on Read operations
•• Optimized Optimized for disk scan over seek for disk scan over seek operationsoperations
•• Storage Storage optimization focused on disk optimization focused on disk scan rate (MB/s)scan rate (MB/s)
StorageStorage
•• Emphasizes Emphasizes transactional transactional performanceperformance
•• Optimized Optimized for disk seek over scan for disk seek over scan operationsoperations
•• Storage Storage optimization focused on optimization focused on I/O operations/s (IOPs)I/O operations/s (IOPs)
Sequential I/O
Sequential I/O
• Scans on large data stores are usually read with sequential read patterns and not random read patterns
• Scalable, predictable performance
Random I/O
• OLTP usually random-read centric. Discrete lookups benefit from index optimization and random read capability.
• Not as predictable & scalable for • Scalable, predictable performance
• Requires 1/3 or fewer drives to match server I/O consumption capability.
• Not as predictable & scalable for data warehousing
• Requires large number of drives to match server I/O consumption capability.
All databases contain both scans and seeks among with other types of reads and writes, DW workload indicate
that the vast majority of reads are sequential – not all
Some SQL Data Warehouses today
Big SAN
Big 64-core Server
Connected together
What’s wrong with this picture?What’s wrong with this picture?
Answer: system out of balance
• This server can consume 16 GB/Sec of IO, but the SAN can only deliver 2 GB/Sec– Even when the SAN is dedicated to the SQL Data
Warehouse, which it often isn’t
– Lots of disks for Random IOPS BUT
– Limited controllers � Limited IO bandwidth– Limited controllers � Limited IO bandwidth
• System is typically IO bound
• Queries are slow
Result: significant investment, not delivering performanceResult: significant investment, not delivering performance
The Alternative: A Balanced System
• Design a server + storage configuration that can deliver all the IO bandwidth that CPUs can consume when executing a SQL Relational DW workload
• Avoid sharing storage devices among servers• Avoid overinvesting in disk drives
– Focus on scan performance, not IOPS
• Layout and manage data to maximize range scan performance and minimize fragmentation
• Layout and manage data to maximize range scan performance and minimize fragmentation
Potential Performance Bottlenecks
FC
HBA
A
B
FC
HBA
A
B FC SWITCH
STORAGE
CONTROLLERA
B
A
BCACHE
SERVER
CACHE
SQL SERVER
WINDOWS
CPU CORES
A
B
DISK DISK
LUN
DISK DISK
LUN
HBA B FC SWITCH
SQL SERVER
CPU CORES
CPU Feed Rate HBA Port Rate Switch Port Rate SP Port Rate
LUN
SQL Server
Read Ahead Rate
LUN Read Rate Disk Feed Rate
SQL Server Fast Track Data Warehouse
• A method for designing a cost-effective, balanced system for Data Warehouse workloads
• Reference hardware configurations developed in conjunction with hardware partners using this
Solution to help customers and partners accelerate their data warehouse
deployments
conjunction with hardware partners using this method
• Best practices for data layout, loading and management
Relational Database Only – Not SSAS, IS, RS
Fast Track Data Warehouse Components
Software:
• SQL Server 2008 Enterprise
• Windows Server 2008
Configuration guidelines:
• Physical table structures• Physical table structures
• Indexes
• Compression
• SQL Server settings
• Windows Server settings
• Loading
Hardware:
• Tight specifications for servers,
storage and networking
• ‘Per core’ building block
Fast Track Scope
Data Path
Analysis Services
Cubes
Reporting Services
Web Analytic Tools
Integration
Services ETL
Supporting Systems BI Data Storage Systems Presentation Layer Systems
Presentation Data
Presentation Data
Data Path
Data Warehouse PerformancePoint
Dedicated SAN,
Storage Array
SharePoint Services
Microsoft Office SharePoint
Data Staging,
Bulk Loading
Reference Architecture Scope (dashed)
Excel Services
Presentation Data
Presentation Data
Two SQL DW Infrastructure Options:
SQL Classic DW or Fast Track SQL DW
Shared Network
Bandwidth
Enterprise Shared
SAN Storage
Dedicated
Network
Bandwidth
SQL Classic DW
Architecture
Leverages Shared SAN
Fast Track SQL DW Architecture
Architecture modeled after DW Appliances
Teradata, DATAllegro..etc “ Appliance Like”
Uses Dedicated SAN arrays and Network
Dedicated SAN
SQL 2008 Data Warehouse
SMP Server
SAN Arrays 1:4 cpu cores
8 Data Disk / Array – 4 Raid 1 Pairs
Simultaneous SQL Server Reads
2 Log and 1 Hot Spare
EMC AX4 – HP MSA2312
IBM 3400
OLTP Applications SQL Fast Track DW supports “Scan Centric” DW
workloads that are index light
Optimizing storage layout for scan
intensive workloads• LUN configuration is based on
RAID1 pairs
– Optimal for scan type access patterns
• Striping across storage is
accomplished via SQL Server data
S
P
A
03 04
RAID GP02RAID GP02
LUN3LUN3
LUN4LUN4
03 04
RAID GP02
LUN3
LUN4
01 02
RAID GP01
LUN1
LUN2
RAID GP03RAID GP03RAID GP03 RAID GP04RAID GP04RAID GP04
09 10
RAID GP05RAID GP05
LUN0
(Logs)
LUN0
(Logs)
09 10
RAID GP05
LUN0
(Logs)
HO
T S
PA
RE
accomplished via SQL Server data
files
• Observed throughput for a single
RAID pair >= 130 MB/s
S
P
B
05 06
RAID GP03RAID GP03
LUN5LUN5
LUN6LUN6
05 06
RAID GP03
LUN5
LUN6
07 08
RAID GP04RAID GP04
LUN7LUN7
LUN8LUN8
07 08
RAID GP04
LUN7
LUN8
HO
T S
PA
RE
Storage Layout Implications for SQL Server
LUN16 LUN 2 LUN 3LUN 1
Permanent FG
Permanent_1.ndf
Perm
anant_DB
Permanent_2.ndf Permanent_3.ndf Permanent_16.ndf
Local Drive 1
Log LUN 1
Permanent DB
Log
TempDB
TempDB.mdf (25GB) TempDB_02.ndf (25GB) TempDB_03ndf (25GB) TempDB_16.ndf (25GB)
Perm
anant_DB
Stage
Database Stage FG
Stage_1.ndf Stage_2.ndf Stage_3.ndf Stage_16.ndf
Stage DB Log
Creating Sequential Data Layout
• Goal: Align logical and physical ordering of data within a Filegroup
• Two primary ways Fast Track optimizes allocation for Sequential Scan– Minimize Fragmentation– Minimize Fragmentation– Manage Load processing
Maximum Consumption RateTheoretical throughput for IO stack
• Using a 2x quad-core server as a building block / starting point
• Ensure that the per-core data consumption rate can
Maximum theoretical throughput for IO stack
components sized for an 8 CPU core Fast Track system
(assumes 200 MB/s per core)
consumption rate can be delivered by all elements of the IO stack
• Sticker on the new car: “Miles Per Gallon”
Server
Windows Server OS
MCR 1.6 GB/s
Storage Enclosure
Storage EnclosureFiber Switch
500 MB/s
500 MB/s
500 MB/s
500 MB/s
300 MB/s
300 MB/s
300 MB/s
300 MB/s
300 MB/s
300 MB/s
300 MB/s
300 MB/s
HBA
HBA
Min2
GB/s
Min2
GB/s
CPU
Socket
(4 Core)
CPU
Socket
(4 Core)
Scaling the IO stackFiber
Switch
Storage Enclosure
Storage Processor
Storage Processor
RAID-1RAID-1RAID-1RAID-1
RAID-1RAID-1RAID-1RAID-1
RAID-1RAID-1
CPU
Socket
(4 Core)
CPU
Socket
(4 Core)
CPU
Socket
(4 Core)
CPU
Socket
(4 Core)
Storage Enclosure
Storage Processor
Storage Processor
RAID-1RAID-1RAID-1RAID-1
RAID-1RAID-1RAID-1RAID-1
RAID-1RAID-1
Storage Enclosure
Storage Processor
Storage Processor
RAID-1RAID-1RAID-1RAID-1
RAID-1RAID-1RAID-1RAID-1
RAID-1RAID-1
Storage Processor RAID-1RAID-1RAID-1RAID-1
RAID-1RAID-1
CPU
Socket
(4 Core)
CPU
Socket
(4 Core)
CPU
Socket
(4 Core)
CPU
Socket
(4 Core)
ServerHBA
HBA
Storage Enclosure
Storage Processor
Storage Processor
RAID-1RAID-1RAID-1RAID-1
RAID-1RAID-1RAID-1RAID-1
RAID-1RAID-1
Storage Enclosure
Storage Processor
Storage Processor
RAID-1RAID-1RAID-1RAID-1
RAID-1RAID-1RAID-1RAID-1
RAID-1RAID-1
Storage Enclosure
Storage Processor
Storage Processor
RAID-1RAID-1RAID-1RAID-1
RAID-1RAID-1RAID-1RAID-1
RAID-1RAID-1
Storage EnclosureStorage Processor
RAID-1RAID-1RAID-1
RAID-1RAID-1
Storage Enclosure
Storage Processor
Storage Processor
RAID-1RAID-1RAID-1RAID-1
RAID-1RAID-1RAID-1RAID-1
RAID-1RAID-1
(4 Core) (4 Core)
HBA
HBA
HBA
HBA
HBA
HBA
Fast Track Data Warehouse Reference
ConfigurationsServer CPU
CPU
CoresSAN Data Drive Count
Initial
Capacity*
Max
Capacity**
HP Proliant
DL 385 G6
(2) AMD Opteron Istanbul
six core 2.6 GHz
12 (3) HP MSA2312fc (24) 300GB 15k SAS 6TB 12TB
HP Proliant
DL 380 G6
(2) Intel Xeon® 5500 Series
Quad core
8 (2) HP MSA2312 (16) 300GB 15k SAS 4TB 8TB
HP Proliant
DL 585 G6
(4) AMD Opteron Instanbul
six core 2.6 GHz
24 (6) HP MSA2312fc (48) 300GB 15k SAS 12TB 24TB
HP Proliant
DL 580 G5
(4) Intel Xeon® 7400 Series six
core
24 (6) HP MSA2312 (48) 300GB 15k SAS 12TB 24TB
HP Proliant
DL 785 G6
(8) AMD Opteron Istanbul
six core 2.8 GHz
48 (12) HP MSA2312 (96) 300GB 15k SAS 24TB 48TB
* Core-balanced compressed capacity based on 300GB 15k SAS not including hot spares and log drives. Assumes 25% (of raw disk space) allocated for Temp DB.** Represents storage array fully populated with 300GB15k SAS and use of 2.5:1 compression ratio. This includes the addition of one storage expansion tray per enclosure.
30% of this storage should be reserved for DBA operations
DL 785 G6 six core 2.8 GHz
Dell PowerEdge
R710
(2) Intel Xeon Nehalem quad
core 2.66 GHz
8 (2) EMC AX4 (16) 300GB 15k FC 4TB 8TB
Dell Power Edge
R900
(4) Intel Xeon Dunnington
six core 2.67GHz
24 (6) EMC AX4 (48) 300GB 15k FC 12TB 24TB
IBM X3650 M2 (2) Intel Xeon Nehalem quad
core 2.67 GHx
8 (2) IBM DS3400 (16) 200GB 15K FC 4TB 8TB
IBM X3850 M2 (4) Intel Xeon Dunnington six
core 2.67 GHz
24 (6) IBM DS3400 (24) 300GB 15k FC 12TB 24TB
IBM X3950 M2 (8) Intel Xeon Nehalem four
core 2.13 GHz
32 (8) IBM DS3400 (32) 300GB 15k SAS 16TB 32TB
Bull Novascale
R460 E2
(2) Intel Xeon Nehalem quad
core 2.66 GHz
8 (2) EMC AX4 (16) 300GB 15k FC 4TB 8TB
Bull Novascale
R480 E1
(4) Intel Xeon Dunnington
six core 2.67GHz
24 (6) EMC AX4 (48) 300GB 15k FC 12TB 24TB
SQL Server Fast Track Data Warehouse 2.0 for
HP – now on G6 PlatformFive AMD and Intel based Reference configurations available for HP:
AMD Based Reference Architectures
2 Processor Configuration
– Server: HP ProLiant DL385 G6 with 2 6-core AMD Opteron CPUs
– Storage server: MSA Storage
– Scalability: 4 – 12 TB
4 Processor Configuration
– Server: HP ProLiant DL 585 G6 with 4 6-core AMD Opteron CPUs
– Storage server: MSA Storage
– Scalability: 12 – 24 TB
8 processor Configuration
– Server: HP ProLiant DL 785 G6 with 8 6-core AMD
Opteron CPUs
– Storage server: MSA Storage
– Scalability: 24 – 48TB
SQL Server Fast Track Data Warehouse 2.0 for
HP – now on G6 Platform
Intel Based Reference Architectures
2 Processor Configuration– Server: HP ProLiant DL380 G6 with 2 4-core Intel Xeon® 5500
Series CPUs
– Storage server: MSA Storage
– Scalability: 4 – 8 TB
4 Processor Configuration– Server: HP ProLiant DL 580 G5 with 4 6-core Intel Xeon®
7400 Series CPUs
– Storage server: MSA Storage
– Scalability: 12 – 24 TB
New Fast Track Data Warehouse 2.0 for IBM
Three Reference configurations available for IBM:2 Processor Configuration
– Server: IBM System x3650 M2 with 2 Quad-core Intel Xeon CPUs– Storage server: IBM System Storage DS3400– Scalability: 4 – 8 TB
4 Processor Configuration– Server: IBM System x3850 M2 with 4 6-core Intel Xeon CPUs– Storage server: IBM System Storage DS3400– Scalability: 12 – 24 TB
8 processor Configuration– Server: IBM System x3950 M2 with 8 Quad-core Intel Xeon CPUs– Server: IBM System x3950 M2 with 8 Quad-core Intel Xeon CPUs– Storage server: IBM System Storage DS3400– Scalability: 16 – 32TB
SQL Server Fast Track Data Warehouse 2.0 for
DELLTwo Reference configurations available for DELL:
2 Processor Configuration
– Server: Dell Power Edge R710 with 2 Quad-core Intel Xeon processors
– 8 CPU Cores
– 32GB Memory– 32GB Memory
– Storage server: EMC CLARiiON AX4
– Scalability: 4 – 8 TB
4 Processor Configuration
– Server: Dell Power Edge R900 with 4 6-core Intel Xeon processors
– 24 CPU Cores
– 96 GB Memory
– Storage server: EMC CLARiiON AX4
– Scalability: 12 – 24 TB
SQL Server Fast Track Data Warehouse for BULL
Two Reference configurations available for BULL:
2 Processor Configuration– Server: Bull Novascale R460 E2 with 2 Quad-core Intel Xeon
processors
– Storage server: EMC CLARiiON AX4
– Scalability: 4 – 8 TB
4 Processor Configuration– Server: Bull Novascale R480 E1 with 4 6-core Intel Xeon
processors
– Storage server: EMC CLARiiON AX4
– Scalability: 12 – 24 TB
• Also included in the Rack:– SQL Server Analysis Services
– SQL Server Reporting Services
– SQL Server Integration Services
– HA Server
– Administration Server (with Management Studio, Backup Server
Fast Track Data Warehouse Benefits
• Lower TCO
– Minimizes risk of overspending on un-balanced hardware configurations
– Commodity Hardware
• Choice• Choice– HW platform
– Implementation vendor
• Reduced Risk
– Validated by Microsoft
– Encapsulates best practices
– Known performance & scalability
SummaryFaster time to solution
High scale: up to 48TB
Low TCO with better price performance; industry standard hardware
Better performance out of the box and predictable performance
Reduced risk through balanced hardware & Best practices
Integration with Madison Hub & Spoke Architecture
Fast Track Dataoffers customers
Twelve reference architectures from HP, Dell, Bull, EMC and IBMSQL Server Fast Track Data
Warehouse has 2 componentsIBM
System Integrators with industry solution templates –Avanade, HP, Hitachi, Cognizant and EMC
Warehouse has 2 components
Next Steps
• Proof Steps
– Quick Start DW Roadmap Service
– Architectural Design Session
– Madison Technology Preview (MTP)
– Review Madison, SQL Server Classic or Fast Track – Review Madison, SQL Server Classic or Fast Track DW HW/SW configurations and pricing
© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market
conditions,
it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation.
MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
top related