Windows AzureData Services

Scott Klueppel

Chief Cloud Evangelist

SOAlutions, Inc.#gwab

@kloopdogg

Agenda - Overview - Management Portal / Tools - Windows Azure Storage - Tables, Blobs, Queues - Windows Azure SQL Database - Putting them together with Cloud Design Pattern Examples

Windows Azure StorageOverviewManagement PortalStorage Emulator / Tools

What is Windows Azure Storage? - Storage as a service – Blobs, Tables, and Queues - Lots of it (up to 200 TB) – pay as you go

- Highly Available - Redundancy

- Highly Scalable - Partitions

- Blazing Speed - Performance “Targets”

Getting Started with Storage - Create a storage account in the portal - Connection string Account Name Account Key

Getting Started with Storage - Use storage emulator for development - Connection string UseDevelopmentStorage=true

Windows Azure SDK

Windows AzureSDK

Visual Studio2013

Visual Studio2012

Visual Studio2010

2.2 Oct2013 Supported Supported Not Supported

2.1 Jul2013 Not Supported Supported Supported

2.0 Apr2013 Not Supported Supported Supported

- SDK contains client libraries - Committed to backwards compatibility

Storage Client Library - SCL 2.1 included with SDK 2.2 - SCL 3.0.x upgrade available with NuGet

Tools to interact with storage Visual Studio Server Explorer CloudXplorer/TableXplorer

AzureXplorer (VS Extension)

DemoManagement Portal

Storage

Tools

Windows Azure StorageTablesBlobsQueues

Tables Concept OverviewAccount Table Entity

gwabjax

racers

results

Name=RickyStatus=ActiveName=DaleStatus=ActiveRace=Jax 500Date=3/29/14

- Structured storage in the form of tables-1MB max size-252 properties

- Quick queries by clustered index (only index)- 2,000 entities per second per partition- Data-defined partition scheme

-Table name + PK

Table Details

- Tables store data more like key-value pairs - No schema

- De-normalized structure optimized for performance

Table Details

First Name Last Name Date of Birth Last Win

Ricky Bobby 1/1/1971

Dale Earnhardt 2/2/1972 Daytona 500

Cal Naughton March 3, 1973

Table Entities - Property Types byte[]

boolDateTimedoubleGuidInt32Int64String

Table Entities - Required properties PartitionKey, RowKey, Timestamp

- Operations on entities Delete Insert Upsert (Insert + Merge/Replace) Update (Merge or Replace) Retrieve Query

Using Tables1. Get table reference

2. Execute operations on entities

CloudStorageAccount storageAccount = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("DefaultConnection"));

CloudTableClient tableClient = storageAccount.CreateCloudTableClient();CloudTable table = tableClient.GetTableReference("racers");

TableOperation upsertOperation = TableOperation.InsertOrReplace(racer);table.Execute(upsertOperation);

Using Tables with DTOs/POCOs1. Get table reference (like before)2. Execute operations on entities via

EntityAdapterWith TableEntity – RacerEntity

var operation = TableOperation.InsertOrReplace(racerEntity);table.Execute(operation);

With DTO – Racer

var adapter = new RacerAdapter(racer);var operation = TableOperation.InsertOrReplace(adapter);table.Execute(operation);

Table Best Practices1. Avoid querying across partitions2. Batch inserts (within partition)

- Order and details in same table/partition

3. Storing aggregate copies - Ensures 1 query 1 partition - Format data up front

4. Use intelligent PKs

DemoTables

Blobs

Queues

Blobs Concept Overview

Account Container Blob

gwabjax

photos

videos

DSC000256.jpg

teamlogo.png

1stPlace.avi

http://<account>.blob.core.windows.net/<container>/<blobname>

Blobs - Types of Blobs - Page - Block

- Operations on blobs Exists List Upload from Stream, ByteArray, Text, File Download to Stream, ByteArray, Text, File Delete

Using Blobs1. Get blob container reference

2. Perform operations with blobs

CloudStorageAccount account = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("DefaultConnection"));

CloudBlobClient client = account.CreateCloudBlobClient();CloudBlobContainer container = client.GetContainerReference("photos");

CloudBlockBlob blockBlob = container.GetBlockBlobReference(refId);blockBlob.UploadFromStream(stream);blockBlob.Metadata["FileName"] = fileName;blockBlob.SetMetadata();

DemoTables

Blobs

Queues

Queues Concept OverviewAccount Queue Messages

gwabjax

orders

emails-to-send

Bulkhttp://...

Smallhttp://...

http://...

Queues - Operations on queues Add Peek Get Update Delete

- Queue naming - All lowercase - Alpha-numeric and hyphen “-” - 3-63 characters

Client Web Role

Storage

Queues in the wildWorker Role

Browser

Web App

Get Survey

CompleteSurvey

Post Results

SurveysQueue

Process

Task

Thank you

Add

Get

Delete Process*

Client Web Role

Storage

Queues in the wildWorker Role

BlobStore

Browser

Web App

Get Survey

CompleteSurvey

Post Results

SmallSurveys

LargeSurveys

SmallTask

BigTask

Small Survey

Get

Delete Process*

AddThank you

Client Web Role

Storage

Queues in the wildWorker Role

Blob/TableStore

Browser

Web App

Get Survey

CompleteSurvey

Post Results

SmallSurveysQueue

LargeSurveysQueue

SmallTask

LargeTask

LargeSurvey

Get

Delete Process*

AddThank you

Upload

Download

Using Queues1. Get queue reference

2. Perform operations with messages

CloudStorageAccount account = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("DefaultConnection"));

CloudQueueClient client = account.CreateCloudQueueClient();CloudQueue queue = client.GetQueueReference("orders");

CloudQueueMessage message = new CloudQueueMessage("ticket-order-17"); queue.AddMessage(message);

DemoTables

Blobs

Queues

Windows Azure SQL Database(formerly SQL Azure)Management PortalData Access

Windows Azure SQL DatabaseTools - SSMS- Visual Studio/SSDT- Modeling toolsCode- ADO.NET- Enterprise Library- ORMs (e.g. Entity Framework, nHibernate)

Key Features- High Availability/Durability

- Three Replicas (1 Primary – 2 Secondary)- 99.9% SLA

- Manageability- Portal, PowerShell, SSMS, REST API

- Predictable Performance- Scale-out*

What’s Different – Maybe Bad?Missing- Extended stored procedures- SQL-CLR- Service Broker- Table partitioning

Important: All tables must have a clustered index

Data Types?Data Type Category Windows Azure SQL Database Support

Exact numerics bigint, bit, decimal, int, money, numeric, smallint, smallmoney, tinyint

Approximate numerics

float, real

Date and time date, datetime2, datetime, datetimeoffset, smalldatetime, time

Character strings char, varchar, text

Unicode character strings

nchar, nvarchar, ntext

Binary strings binary, varbinary, image

Spatial data types geography, geometry

Other data types cursor, hierarchyid, sql_variant, table, timestamp, uniqueidentifier, xml

Development Story- Local SQL 2012/Express database - SSDT Database projects

- Deploy directly to Azure from Visual Studio- Create deployment package- Use Release Management

- Azure Connection String<add name="DefaultConnection"

connectionString="Server=tcp:[server].database.windows.net,1433;Database=[database]; User ID=[database]-host@[server]; Password=[password];Encrypt=true;"

providerName="System.Data.SqlClient"/>

DemoManagement Portal

Create Server

Create Database

Firewall Rules

Patterns at Work• Cache-aside Pattern• Materialized View Pattern• Competing Consumers Pattern• Compensating Transaction Pattern

Cache-Aside Pattern- Build cache on-demand

- Read cache first- If not there

- Get from data store- Store in cache

Materialized View Pattern- Improve performance in systems with

difficult queries- Pre-populated views - Not updated by app, can be entirely rebuilt- Inherent delays- Doesn’t query well

Competing Consumers Pattern- Protect against a large influx (burst) of

requests- Balanced workload- Scalable

Compensating Transaction Pattern- Eventual consistency

- Series of autonomous steps- Intermediate steps appear inconsistent

- Compensating Transaction- Intelligent process to undo succeeded steps

Compensating Transaction Pattern- Example: Itinerary Creation

- Book 3 flights and 2 hotel rooms- If one or more are unavailable, we start

over

Compensating Transaction Pattern- Example:

Itinerary Creation

What did we learn?- Azure data storage options/uses- Each has pros/cons/best fit- Management Portal and Tools- Design patterns

Questions?

Scott Klueppel

Chief Cloud Evangelist

SOAlutions, Inc.#gwab

@kloopdogg

Thank you!

Scott Klueppel

Chief Cloud Evangelist

SOAlutions, Inc.#gwab

@kloopdogg

References• Cloud Design Patterns (P&P)

http://msdn.microsoft.com/en-us/library/dn568099.aspx

• Data Services – Storagehttp://msdn.microsoft.com/en-us/library/windowsazure/gg433040.aspx

• Storage Differences – Emulator vs Cloudhttp://msdn.microsoft.com/en-us/gg433135

• Data Services – SQL Databasehttp://msdn.microsoft.com/en-us/library/windowsazure/ee336279.aspx