extreme scaling with sql azure sql bits 7, york, october 2010 martin schmidt – miracle a/s denmark
TRANSCRIPT
ExtremeScaling with SQL
AzureSQL Bits 7, York, October 2010Martin Schmidt – Miracle A/S
Denmark
Subtitle:The Safetrack story
AgendaO Speaker InfoO Safetrack facts & system infoO Safetrack architectureO Database designO Issues with current setupO The Azure platformO ShardingO Solution WalkthroughO Moving Exsisting Data to SQL Azure
Speaker infoO Name: Martin SchmidtO Email: [email protected] Blog: www.performanceduo.comO Twitter: @ms1333
AgendaO Speaker InfoO Safetrack facts & system infoO Safetrack architectureO Database designO Issues with current setupO The Azure platformO ShardingO Solution WalkthroughO Moving Exsisting Data to SQL Azure
Safetrack factsO Danish Company – started 2005O GPS tracking of trucks and excavatorsO Started out as an insurance service
against theftO Today: A complete platform for fleet
managementO Tracks more than 25.000 unitsO Website: http://www.safetrack.dk
Unit Types
How does this Work?
How does this WorkO All units are equipped with a GPS
transmitterO Sends binary packages over GPRSO Built-in SIM cardO Sends a package every 2 min when
runningO Sends a package every 15 min when
parkedO Data is sent using UDP Protocol
ME - 301
Length = 75 mm Width = 45 mmHeight = 18 mm
Package dataO ReporttimeO LongitudeO LatitudeO SpeedO KM StatusO Inputs status information
O Is the lift up / downO Is the engine running
O Timecounter info on inputsO For how long has the engine been runningO How long has the lift been in use
AgendaO Speaker InfoO Safetrack facts & system infoO Safetrack architectureO Database designO Issues with current setupO The Azure platformO ShardingO Solution WalkthroughO Moving Exsisting Data to SQL Azure
System Archietecture
AgendaO Speaker InfoO Safetrack facts & system infoO Safetrack architectureO Database designO Issues with current setupO The Azure platformO ShardingO Solution WalkthroughO Moving Exsisting Data to SQL Azure
Database Design
AgendaO Speaker InfoO Safetrack facts & system infoO Safetrack architectureO Database designO Issues with current setupO The Azure platformO ShardingO Solution WalkthroughO Moving Exsisting Data to SQL Azure
Issues ?O Database and application on same
boxO Everything runs on the same single
serverO Single Point of Failure
O HardwareO DatabaseO Services / Website
O No HA solutionO Limited scaling options (Buy bigger
HW)
Issues ?O All elements are independendO Easy to split thingsO No ”secret” data
O No personal or banking informationO Not breaking the Danish data law
Issues ?O SQL Azure in the future?O Windows Azure in the future?
O Why Not
AgendaO Speaker InfoO Safetrack facts & system infoO Safetrack architectureO Database designO Issues with current setupO The Azure platformO ShardingO Solution WalkthroughO Moving Exsisting Data to SQL Azure
SQL AzureO High Availibity = Free
O 1 DB = 3 DB’s (automatic failover)O Size limitations
O 1 – 50 GBO Safetrack database > 250 GB
O Get the database off the current boxO Release more power to the Windows
service
SQL AzureO Let Microsoft be your primary DBA!O SQL Azure = Black boksO Running your database with other
unknown applications
Windows AzureO Moving the website to Windows
AzureO Easy peasy
O The Windows Service is not compatible with Azure at presentO No support for UDP – TCP Only
SQL AzureO Lets move the database to the Cloud
O Latency from the local Windows service to SQL Azure is not a big issue.
O Size is a issue, we will have to Shard the data across multiple databases
O Sharding gives us scalability
AgendaO Speaker InfoO Safetrack facts & system infoO Safetrack architectureO Database designO Issues with current setupO The Azure platformO ShardingO Solution WalkthroughO Moving Exsisting Data to SQL Azure
Sharding factsO Partition data across multiple databasesO Each database holds a subset of the dataO A query needs only to look in one
databaseO Sharding is no silver bullet, you still need
to thinkO Important to choose the correct sharding
keyO Try to avoid fan-out queries, if possible
AgendaO Speaker InfoO Safetrack facts & system infoO Safetrack architectureO Database designO Issues with current setupO The Azure platformO ShardingO Solution WalkthroughO Moving Exsisting Data to SQL Azure
Solution walkthroughO Shard key candidates
O UnitIDO CustomerID
O The winner is:O CustomerID
O All reports on the web front end is based on customerID. This gives the customer the option to se the status of all his units.
Data SyncronisationO MS Data sync Framework
O Requires SQL Agent to do scheduling
O SQL Azure Data Sync (Dev Preview)O The winner so farO We will have a look in a minuteO Pretty Slow
O SQL Server Integration ServicesO Still no scheduler
O Windows Azure worker roleO We need to test this
Database Design
DEMOO Enough Power Pointing
AgendaO Speaker InfoO Safetrack facts & system infoO Safetrack architectureO Database designO Issues with current setupO The Azure platformO ShardingO Solution WalkthroughO Moving Exsisting Data to SQL Azure
Loading data into Azure
O Task: Load existing data into SQL AzureO 265.000.000 Rows (ReportMessage)O 150 GB
O Possible solutionsO BCP UtilityO SSIS
Loading data into Azure
Loading data into Azure
O Slow, poor performanceO Still working on improving
performance
O Tested throughput (Laptop)O 1000 Rows / sec
SQL Azure RessourcesO SQL Azure webfrontendO http://sql.azure.com
O SQL Azure LabsO http://www.sqlazurelabs.com
O SQL Azure team BlogO http://blogs.msdn.com/b/sqlazure/
More RessourcesO PerformanceDUO
O http://www.performanceduo.com
O Miracle A/SO http://www.miracleas.dk
O MiracleDBO http://Miracledb.codeplex.com
O SafetrackO http://www.safetrack.dk
Q & AO Questions?
Thank You
Coming up…P/X001Understanding and Preventing SQL Injection AttacksKevin KlineP/L001SSIS FieldnotesDarren GreenP/L002The (Geospatial) Shapes of Things to ComeSimon MunroP/L005End to End Master Data Management with SQL Server Master Data ServicesJeremy KashelP/T007Understanding Microsoft Certification in SQL ServerChris Testa-O'Neill
#SQLBITS