vld bs conquering the giant
DESCRIPTION
TRANSCRIPT
![Page 1: Vld bs conquering the giant](https://reader033.vdocuments.mx/reader033/viewer/2022061215/549ecf4eac795947768b48a1/html5/thumbnails/1.jpg)
© 2008 Quest Software, Inc. ALL RIGHTS RESERVED.
Conquering the Giants:Very Large Databases
For audio, call the phone number in your meeting invite.
Audio will not come through the computer speakers.
![Page 2: Vld bs conquering the giant](https://reader033.vdocuments.mx/reader033/viewer/2022061215/549ecf4eac795947768b48a1/html5/thumbnails/2.jpg)
• SQL Server Expertfor Quest Software
• Editor-in-Chief at SQLServerPedia.com
• Former SQL DBA• Managed data
warehouses,>80tb SAN storage for $7b company
![Page 3: Vld bs conquering the giant](https://reader033.vdocuments.mx/reader033/viewer/2022061215/549ecf4eac795947768b48a1/html5/thumbnails/3.jpg)
Conquering the Giants: Agenda• What’s a VLDB?• Ginormous Growth• Big Backups• Tremendous Tables• Slow SQL Statements• Your Wingman: Quest• Resources
Photo Licensed with Creative Commons from http://www.flickr.com/photos/pagedooley/2172001078/
![Page 4: Vld bs conquering the giant](https://reader033.vdocuments.mx/reader033/viewer/2022061215/549ecf4eac795947768b48a1/html5/thumbnails/4.jpg)
What Makes a VLDB?• 1 terabyte• Billion rows
• But Not:User Qty,Criticality,Transactions Per Second
4
Photo Licensed with Creative Commons from http://www.flickr.com/photos/tcmhitchhiker/1053095394
![Page 5: Vld bs conquering the giant](https://reader033.vdocuments.mx/reader033/viewer/2022061215/549ecf4eac795947768b48a1/html5/thumbnails/5.jpg)
5
VLDB Space Challenges
![Page 6: Vld bs conquering the giant](https://reader033.vdocuments.mx/reader033/viewer/2022061215/549ecf4eac795947768b48a1/html5/thumbnails/6.jpg)
Default File Growth Settings
6
![Page 7: Vld bs conquering the giant](https://reader033.vdocuments.mx/reader033/viewer/2022061215/549ecf4eac795947768b48a1/html5/thumbnails/7.jpg)
Starts Out Small…
7
![Page 8: Vld bs conquering the giant](https://reader033.vdocuments.mx/reader033/viewer/2022061215/549ecf4eac795947768b48a1/html5/thumbnails/8.jpg)
Ginormous Growths
8
![Page 9: Vld bs conquering the giant](https://reader033.vdocuments.mx/reader033/viewer/2022061215/549ecf4eac795947768b48a1/html5/thumbnails/9.jpg)
Instant File Init in Secpol.msc
9
![Page 10: Vld bs conquering the giant](https://reader033.vdocuments.mx/reader033/viewer/2022061215/549ecf4eac795947768b48a1/html5/thumbnails/10.jpg)
VLDBs = Big Backups• Time Problem• Nightly Load Windows• Time to Recover• Dev, QA, Test, DR
10
![Page 11: Vld bs conquering the giant](https://reader033.vdocuments.mx/reader033/viewer/2022061215/549ecf4eac795947768b48a1/html5/thumbnails/11.jpg)
Giant Monstrous Backups
11
![Page 12: Vld bs conquering the giant](https://reader033.vdocuments.mx/reader033/viewer/2022061215/549ecf4eac795947768b48a1/html5/thumbnails/12.jpg)
Cutting Them Down to Size
12
![Page 13: Vld bs conquering the giant](https://reader033.vdocuments.mx/reader033/viewer/2022061215/549ecf4eac795947768b48a1/html5/thumbnails/13.jpg)
Next Weapon: SAN Snapshots
13
![Page 14: Vld bs conquering the giant](https://reader033.vdocuments.mx/reader033/viewer/2022061215/549ecf4eac795947768b48a1/html5/thumbnails/14.jpg)
Post-Snapshot
14
![Page 15: Vld bs conquering the giant](https://reader033.vdocuments.mx/reader033/viewer/2022061215/549ecf4eac795947768b48a1/html5/thumbnails/15.jpg)
Making the Snap Available
15
![Page 16: Vld bs conquering the giant](https://reader033.vdocuments.mx/reader033/viewer/2022061215/549ecf4eac795947768b48a1/html5/thumbnails/16.jpg)
Weapon: Backup Compression
16
Photo Licensed with Creative Commons from http://www.flickr.com/photos/tcmhitchhiker/1053045338/
![Page 17: Vld bs conquering the giant](https://reader033.vdocuments.mx/reader033/viewer/2022061215/549ecf4eac795947768b48a1/html5/thumbnails/17.jpg)
Or – Compress Everything!
17
![Page 18: Vld bs conquering the giant](https://reader033.vdocuments.mx/reader033/viewer/2022061215/549ecf4eac795947768b48a1/html5/thumbnails/18.jpg)
VLDB Problem: Giant Tables
18
![Page 19: Vld bs conquering the giant](https://reader033.vdocuments.mx/reader033/viewer/2022061215/549ecf4eac795947768b48a1/html5/thumbnails/19.jpg)
Hunt Down and Kill Unused Indexes
19
![Page 20: Vld bs conquering the giant](https://reader033.vdocuments.mx/reader033/viewer/2022061215/549ecf4eac795947768b48a1/html5/thumbnails/20.jpg)
Breaking Giants Into Pieces
20
![Page 21: Vld bs conquering the giant](https://reader033.vdocuments.mx/reader033/viewer/2022061215/549ecf4eac795947768b48a1/html5/thumbnails/21.jpg)
The New Way: Partitioning
21
![Page 22: Vld bs conquering the giant](https://reader033.vdocuments.mx/reader033/viewer/2022061215/549ecf4eac795947768b48a1/html5/thumbnails/22.jpg)
Partitioning Basics• Create filegroups• Create files in the filegroups• Design partition function• Design partition scheme• Alter table’s clustered index
22
![Page 23: Vld bs conquering the giant](https://reader033.vdocuments.mx/reader033/viewer/2022061215/549ecf4eac795947768b48a1/html5/thumbnails/23.jpg)
Sliding Window Loads• Partition by day• Create new
table similarto Sales table,on same filegroup
• Load one day’sdata into empty table
• Swap out a partition for that table
23
![Page 24: Vld bs conquering the giant](https://reader033.vdocuments.mx/reader033/viewer/2022061215/549ecf4eac795947768b48a1/html5/thumbnails/24.jpg)
Giant Drawbacks• Not easy to change• Requires lots of
experimentation• Partition elimination
doesn’t always work• Requires familiarity
with queries, data• Storage skills help• Hardware-dependent
24
Photo Licensed with Creative Commons from http://www.flickr.com/photos/cellphonesusie/3224335140/
![Page 25: Vld bs conquering the giant](https://reader033.vdocuments.mx/reader033/viewer/2022061215/549ecf4eac795947768b48a1/html5/thumbnails/25.jpg)
SQLCAT MAXDOP FAQ
25
![Page 26: Vld bs conquering the giant](https://reader033.vdocuments.mx/reader033/viewer/2022061215/549ecf4eac795947768b48a1/html5/thumbnails/26.jpg)
The New-New Way: 2008 R2
26
![Page 27: Vld bs conquering the giant](https://reader033.vdocuments.mx/reader033/viewer/2022061215/549ecf4eac795947768b48a1/html5/thumbnails/27.jpg)
Slow SQL Statements
27
![Page 28: Vld bs conquering the giant](https://reader033.vdocuments.mx/reader033/viewer/2022061215/549ecf4eac795947768b48a1/html5/thumbnails/28.jpg)
Your Shovel: Resource Governor• Slows zombie
queries down• Throttle CPU,
memory• Can throttle jobs,
backups too• Doesn’t throttle
storage (yet)
28
Photo Licensed with Creative Commons from http://www.flickr.com/photos/archiemcphee/3612278108/
![Page 29: Vld bs conquering the giant](https://reader033.vdocuments.mx/reader033/viewer/2022061215/549ecf4eac795947768b48a1/html5/thumbnails/29.jpg)
Implementing The Governator• Workload groups:
– ETL processes– Reports– AdHoc
• Resource pool:– ETL– Reports
• Classification function based on:– Login name– Application name
29
Photo Licensed with Creative Commons from http://www.flickr.com/photos/shaunwong/2467881648/
![Page 30: Vld bs conquering the giant](https://reader033.vdocuments.mx/reader033/viewer/2022061215/549ecf4eac795947768b48a1/html5/thumbnails/30.jpg)
End Results in SSMS 2008
30
![Page 31: Vld bs conquering the giant](https://reader033.vdocuments.mx/reader033/viewer/2022061215/549ecf4eac795947768b48a1/html5/thumbnails/31.jpg)
Resource Governor Caveats• Ask the network team about QoS• Get manager consensus on SLAs• Keep it absolutely transparent• Avoid false alarms about performance
problems• Teach managers how to “see” it working
31
![Page 32: Vld bs conquering the giant](https://reader033.vdocuments.mx/reader033/viewer/2022061215/549ecf4eac795947768b48a1/html5/thumbnails/32.jpg)
Your Wingman: Quest Software• Capacity Manager v3.0
– Capacity Planning– Index Defragmentation– Partitioning GUI
• LiteSpeed v5.1– SmartDiffs– Throttled Compression
32
![Page 33: Vld bs conquering the giant](https://reader033.vdocuments.mx/reader033/viewer/2022061215/549ecf4eac795947768b48a1/html5/thumbnails/33.jpg)
Showing the Giant Who’s Boss• Use Instant File Initialization• Consider Differential Backups• Compress Data• Drop Baggage• Partitioning FTW• Resource
Governator• Check Out Quest
Capacity Manager
Photo Licensed with Creative Commons from http://www.flickr.com/photos/keeg/1661401660/
![Page 34: Vld bs conquering the giant](https://reader033.vdocuments.mx/reader033/viewer/2022061215/549ecf4eac795947768b48a1/html5/thumbnails/34.jpg)
VLDB Resources• Presentation Resources:
http://sqlserverpedia.com/wiki/VLDB
• Quest Software:http://www.quest.com/sql-server/
• Upcoming Webcasts:http://www.quest.com/events