to blob or not to blob
DESCRIPTION
TRANSCRIPT
To BLOB or Not to BLOB?Dan Holme
Chief SharePoint Evangelist
[email protected]@danholme
Randy WilliamsEnterprise Trainer & Evangelist
[email protected]@tweetraw
© 2011 AvePoint, Inc. All rights reserved. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of
AvePoint, Inc.
Agenda
What are BLOBs?How BLOBs affect SharePoint storageOptions for externalizing BLOBsBenefits & drawbacks of externalizationUnderstanding the Big PictureFILESTREAM provider vs third partyQ&A
Content lifecycle
Birth(Creation)
Life(Use)
Death(Dispositio
n)
BLOBs
Documents, databases, and BLOBs
SharePoint
SQL Server 2008/R2
SharePoint Documents in SQL
DEMO
BLOBs in the AllDocStreams table
“”
Typically, as much as 80 percent of data for an enterprise-scale deployment of SharePoint Foundation consists of file-based data streams that are stored as BLOB data. These BLOB objects comprise data associated with SharePoint files.
msdn.microsoft.com/en-us/library/bb802976.aspx
The burden of BLOBs
The SharePoint storage dilemma
Documents, databases, and BLOBsStorage growth
SharePoint
SQL Server 2008/R2
Content Database
Content Database
Content DatabaseYear 0 Content
Year 1Collaboration
Year 2Content Mgt
Year3Records Mgt
Active Content Actual Content
Planning for SharePoint storage
Planning for implementation and future growth
Don’t forget about: x 3.5• Versioning• SharePoint Recycle Bin• SharePoint Indexes• SQL Server Transaction Logs = 1TB
From: Physical Storage Recommendations (SharePoint Server)http://technet.microsoft.com/en-us/library/cc298801.aspx
10
Content scaling support & guidanceSharePoint 2010 RTM
Content Database200 GB (collaboration)1 TB (Records Center)
Site Collection100 GB200 GB (1 site in a CDB)
July 2011
Content Database200 GB (out-of-box)4 TB (collaboration)*Unlimited (archive)*
Site Collection 100 GB (out-of-box)Up to size of CDB*
Items per CDB60 million
* Conditions apply
QHow big are your biggest content databases?
0-100 GB100-200 GB200-500 GBMore than 500 GBI have no idea
QShould I externalizeBLOBs?
Advantages of keeping BLOBs in SQL
One storage container toMaintainMonitorRecover
Tier I storagePerformance relative to lower tiers of storagebenefits all content access
SQL cachingPerformance of reads/writes of small documentsSQL Caching benefits reads
Disadvantages of large CDBs & BLOBs
Storage costContent
Current versionPrevious versions
Transaction logsBackups
SLA timeframesBackupRecovery
SQL operationsLockingReindexing
PerformanceSlower reads with large documentsBLOBs written twice
Overview of BLOB externalization
Web Front-end
User
Upload
Poin
ter
(stu
b)
Disk Storage
DatabaseMetadata
File
EB
S/R
BS BLOB
Options for externalizing
External BLOB Storage (EBS)Released with SharePoint 2007 SP1Supported by SharePoint 2010
Remote BLOB Store (RBS)Released in SQL Server 2008 R2 Feature Pack
Can be installed on SQL Server 2008 SP1
Supported by SharePoint 2010
Both are interface specificationsNeed provider to communicate with BLOB store
SharePoint Documents in SQL
DEMO
Externalized BLOBs in AllDocStreams
BLOB store
Externalized BLOBs are transparent
Check-in/Check-outVersioningOffice applicationsSearch crawling (indexing)WorkflowsAlertsAnything using the SharePoint API
Third party toolsCustom code
Storage Cost
Performance
Advantages of externalizing
Storage CostPerformance
“”
Maintaining large quantities of BLOB data in a SQL Server database is a suboptimal use of SQL Server resources. You can achieve equal benefit at lower cost with equivalent efficiency by using an external data store to contain BLOB data.
msdn.microsoft.com/en-us/library/bb802976.aspx
Why Are BLOBS a Problem?
Storage Cost
Performance
Advantages of externalizing
Storage CostPerformance
Read*Large documents
Write*All other activity by users of the CDB and SQL serverSLA timeframes
* Dependent on performance characteristics of BLOB store
What’s the performance gain?
Source: SQL Server RBS Performance with SharePointMicrosoft Download Center
SQL BLOB RBS Gain
Database Size - 1 TB 2292 GB 26 GB 98.9%
Database Backup Size - 100 GB 217 GB 7 GB 96.8%
Database Backup Time - 217 GB 2490 sec 38 sec 98.5%
Database Defrag Time - 100 GB 120 sec 4 sec 96.7%
Avg. SharePoint Response Time
28 msec 21 msec 25.0%
Large File Upload - 500 MB 55 seconds
29 seconds
47.6%
Storage Cost
Performance
Advantages of externalizing
Storage CostPerformance
Read*Write*All other activity by users of the CDB and SQL serverSLA timeframes
Access to features of BLOB storage platformEfficient content restructure
EBS and RBS
EBS RBSAPI SharePoint SQL + SharePoint client
SharePoint version WSS v3 or MOSS 2007 SP1 or later
SharePoint 2010, v.Next
SQL version Any SQL Server 2008 or R2
Externalization rules File size and type File size
Scope Site collection Content database
Microsoft support Deprecated Likely to continue
Provider Third party SQL FILESTREAMThird party
Choosing third-party externalization
PerformanceBLOB store platform
File system, SAN, NASShared folderCloudIntegration with your storage platform
Externalization rulesManageability
Backup, recoveryHigh availabilityDisaster recoveryLong term retention, archiving & tiered storage
Cost
SharePoint Uncensored
All things considered
Storage Cost
Performance
Maintenance and
Administration
Backup & Recovery
SLAs (RTO/RPO)
High Availability &
Disaster Recovery
Updates and Upgrades
Migration
Archiving and Disposition
Content lifecycle
Birth(Creation)
Life(Use)
Death(Dispositio
n)
Content lifecycle meets RBS
Storage Cost
Migration
Archiving and
disposition
RBS determines whether BLOB gets externalized to another storage platform on uploadArchiving & disposal
Third party utilities can add business rules to externalization
Externalization: Support and storage
FEATURE
Externalize BLOBs to DAS, iSCSI NAS/SAN
Externalize BLOBs to file share, WORM
Externalize to the Cloud (Azure, Amazon etc)
Native compression and encryption
FILESTREAM
THIRD PARTY
Externalize to multiple storage providers within 1 CDB
SharePoint 2010 (Server and Foundation)
SharePoint 2007 (MOSS 2007 and WSS v3)
All SQL versions (2000, 2005, 2008, 2008 R2)
All SQL editions (Express/Standard/Enterprise)
Externalization: Backup, recovery, DR
FEATURE
Synchronous backup of BLOB store & SharePoint
Backup of content DB independent of BLOB store
Item level Recovery
Platform Level Recovery
Restore without DB staging
FILESTREAM
THIRD PARTY
Externalization: Content lifecycle support
FEATURE
Content restructure (shallow copy) across Web apps
Content replication
Connect to and manage file shares through SharePoint
Connect to and manage media shares through SharePoint
Business rule support (content type, metadata, access date)
Externalize to hardware-based HSM
FILESTREAM
THIRD PARTY
Resources
SharePoint Team Blog: Data Storage Changes in SP1
http://tinyurl.com/3rlvfnp
MSDN: External Storage of Binary Large Objects (BLOBs) in SharePoint Foundation
http://tinyurl.com/yay545y
White paper: SQL Server RBS Performance with SharePoint Server 2010…
http://tinyurl.com/3ccucww
© 2011 AvePoint, Inc. All rights reserved. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of
AvePoint, Inc.