2

Microsoft project code named

for Scalable and Available Applications

Anil Nori, Distinguished Engineer Muralidhar Krishnaprasad (“MK”), Principal ArchitectMicrosoft

Session Code: DAT202

“Velocity”

3

Outline

Need for Distributed CachesUsage ScenariosProject “Velocity” - Introduction and VisionKey ConceptsAPIsCache Configurations ArchitectureRelease ScheduleDemos

4

• Rapidly dropping memory prices, enabling huge amounts of data to reside in-memory• Availability of cheap commodity hardware that can be easily replaced• Multi-core CPUs, cheap flash memory

Most composite applications designed for serious scale and availability are using mid-tier, distributed caching technologies today.

• Data is sourced from all over the place: local, federated, cloud

• An abundance of applications working together, each with different shapes for the same data

• More workloads require increasing scale, availability, and performance as they become mission critical

• Applications must be cheap to scale

Evolving requirements• Logic and data distributed, with data

living closer to mid-tier logic• Global shared data structures enable

application productivity and acceleration

• Recognition of different types of data with different semantics

• Deployment of high-speed data storage where feature set is not “one size fits all”

Evolving application architecturesbeing solved by

Underlying trends

enabled by

5

Distributed Cache UsageH

oriz

onta

lVe

rtica

ls

Scenario

Web

• User-specific HTTP session and shared state across web farm• In-flight shopping carts for web retail• Enabling online self-service applications• Explicit storage of pre-computed or highly-accessed data

LOB • Enterprise-wide product catalog for POS, analytics• Caching frequently used reference data for a ERP application

Telco • Cellular/VOIP: compute utilization, prepay charges, call routing and session info • SMS: message content / notification / receipt, billing

Travel • Aggregated flight pricing / availability retrieved from airlines

Defense • Sensor network data processing and threat detection

Financial• Per-user portfolio data and delayed quote storage for trading

• Aggregate and process ticker stream for algorithmic trading

6

A version of the authoritative dataAggregated or transformed

Each version is uniqueRefreshed periodically Examples

Web and Enterprise (Product) CatalogsUser, Employee data

Access patternMostly readShared & Concurrent Access

ScaleLarge number of accesses

FunctionalityKey based AccessQuery & FilteringLoading

Caching Reference Data

Web Tier

Clients

Local Cache (in Proc)

Distributed Cache Servers

Data Tier

Usernames, Name-> ID Mapping

Friend Lists Usernames

Scenario: Social Networking

7

Data typically generated as part of the application activityActive during business transactions

Typically logged to a backend data sourceHistorical data

Examples Shopping Cart Session StateEnterprise LOB app (Purchase Order)

Access patternRead and writePrimarily exclusive access

ScaleHigh data (and access) scale

FunctionalityKey based accessTransactions

Caching Activity-oriented Data

Data Tier

Mid Tier

Rich Clients

Distributed Cache

IntegrationHub

Thin Clients

Web Tier

Vendor services,

Pricing

VendorSources

Order, Invoice,

Payment

External Systems…

Aggregated Vendor

Catalogs

Scenario: Enterprise LOB Application

8

Caching Resource-oriented Data

Authoritative dataModified by transactions; spans transactionsExamples

Flight Inventory

Access pattern: Read and writeShared access

FunctionalityKey based accessTransactions

ScaleLarge number of concurrent accessesRelaxed consistency for scale

Scenario: Flight Inventory and Pricing

App Logic

Continental

American

United

Distributed Cache

Flight Routing Itinerary

Flight Segment Flight Price

Airli

nes

Inventory

Booking Service

9

Application Cache Requirements

PerformanceMillisecond access10s of 1000s of accesses

Scale10s – 100s of nodes in enterprise100s – 1000s in web applications

FunctionalityKey based and Query based accessTransactions, Optimistic concurrency controlInvalidations

AvailabilityCertain enterprise and e-commerce data (e.g. shopping cart, inventory)

ConsistencyRelaxed consistency for scale and performance

10

Unified Cache View

What is Velocity?An explicit, distributed, in-memory application cache for all kinds of data (CLR objects, rows, XML, Binary data etc.)

Fuse "memory" across machines into a unified cacheClients can be spread across machines or

processes

Clients Access the Cache as if it

was a large single cache

Cache Layer distributes data

across the various cache

nodes

11

Where does Velocity fit?

App (Mid) TierIIS apps, ASP.NET, .NET Apps, PHP Apps

etc.

Cache TierVelocityData Tier

SQL Server, SSDS, etc.

ClientDesktop, Mobile, etc.

Web ClientAJAX, Silverlight etc.

12

Velocity Concepts

13

HostPhysical processes hosting Velocity instance.

Named CachesCan span across machinesDefined in the configuration file

RegionsPhysically co-located Container of Cache ItemsMay be implicit or explicitly created

Cache ItemKey, Payload (Object ), Tags, TTL, Timestamps, Version

Velocity Logical HierarchyMachine 2Machine 1

Cache Host ACache Host A Cache Host BCache Host B Cache Host C Cache Host C Machine 3

Cache Host DCache Host D

Named Cache : Product Catalog

Named Cache : Electronics Inventory

Regions Region A

Key Payload Tags Key Payload Tags 121 xxxx “Toy” “Child”

123 yyyy “Toy” “Chair”..

Machine -> Cache Host -> Named Caches -> Regions -> Cache Items -> Objects

14

Velocity – Example Usage// Create instance of cachefactory (reads appconfig)CacheFactory fac = new CacheFactory();

// Get a named cache from the factoryCache catalog = fac.GetCache("catalogcache");

// ------------------------------------------------------// Region based Get/Putcatalog.CreateRegion("toyRegion");

// Both toy and toyparts are put in the same region catalog.Put("toyRegion", "toy-101", new Toy( .,.));Catalog.Put("toyRegion", "toypart-100", new ToyParts(…));

Toy toyObj = (Toy)catalog.Get("toyRegion", "toy-101");

//-------------------------------------------------------// Simple Get/Putcatalog.Put("toy-101", new Toy("thomas", .,.));

// From the same or a different clientToy toyObj = (Toy)catalog.Get("toy-101");

15

Velocity Access APIsSimple Access

Get – Returns object or entire Cache item

Add - Creates new entry else exception if entry exists

Put - Replaces existing entry or creates a new one

Remove - Removes existing entry

Tag Based AccessSearch limited to a specific regionSearch on "all" or "any" equality match of tags

EnumerationEnumerate entire region or result from tag search

16

Velocity Client Demo

demo

17

Types of CachesDistributed Configurations

Partitioned CacheData Partitioned across all nodes in the named cacheUsed for Scale & Availability

Replicated CacheData replicated across all nodes of the named cacheUsed for scaling (read) requests

Local CacheCache in the application process space for frequently accessed itemsPayload kept in object form

18

Velocity Deployment

Velocity Cache Service

Velocity Cache Service

Cluster Service

Application ApplicationVelocity Client

Local CacheVelocity Client

Local Cache

Caching Tier

ApplicationTier

19

Velocity Client2Velocity Client1

Partitioned Cache Put (K2, V2) on Client1

Routing Layer routes to Cache2 (primary)Cache2 performs local operations & returns back to client

Get (K2) on Client2Client2 routes to Cache2 for K2 & gets value from there

Put(K2, v2) Get(K2)

Routing layer Routing layer

Cache2Cache1

Primary Regions

K2, V2

Primary Regions

K1, V1

Cache3

Primary Regions

K3, V3

20

Velocity Client

Local Cache

Velocity ClientLocal Cache

Partitioned Cache fronted by Local Cache

Local Cache can help speed up access on clients

Put(K2, v2) Get(K2)

Routing layer Routing layer

Cache2Cache1

Primary Regions

K2, V2

Primary Regions

K1, V1

Cache3

Primary Regions

K3, V3

K2, V2K2, V2

21

Achieving Scale and PerformanceScale on Data Size

More machines => More memory to cacheScale on Cache Throughput

Cache Throughput: Operations/sec from entire cacheMore machines => keys distributed across more machines => better throughput

PerformanceHigh Performance by scaling out data and processingIncreased memoryIncreased processing

Workload distributed across multiple cache nodes

22

Velocity Scale Demo

demo

23

Basic Layout

Load Test Controller

…

Server 1 Server 2 Server 3

Cache Client1 Cache Client 2 Cache Client n

Load Controller Spawns Test

Clients

Client Machines run

test

Cache Service Cluster

Partitioned Cache

24

Demo Output

Single ServerThroughput Increases with Increasing Load

Until Server Saturation

Load

Throughput

Latency

Server 2 AddedThroughput Increases

Latency DecreasesUntil Server Saturation

Server 3 AddedThroughput Increases

Latency Decreases

25

Concepts Contd..

26

Velocity ClientVelocity Client

Cache2

Secondary Regions

Cache1

Secondary Regions

Cache3

Secondary Regions

Primary Regions

Availability

Put(K2, v2)

K2, V2

K2, V2

Get(K2)

Primary Regions Primary Regions

K3, V3 K1, V1 K1, V1K2, V2K3, V3

K3, V3K1, V1

Routing layer Routing layer

Put (K2, V2) on Cache1Cache2 replicates changes to secondaries Cache1 and Cache3Waits for acks from Cache1 and Cache2; returns back to client

Get (K2) on Cache3Cache3 routes to Cache2 primary for K2 & gets value from there

27

Cache2Cache1 Cache3

Replicated Regions

Replicated CachePut (K2, V2) on Cache1

Cache2 performs local operation & Acks back to Cache1Cache2 Asynchronously replicates changes to all machines

Get on cache3Cache 3 reads from local store and returns the result

Put(K2, v2)

K2, V2

Get(K2)

Replicated Regions Replicated Regions

K3, V3 K1, V1 K1, V1K3, V3 K3, V3K1, V1

K2, V2K2, V2

Routing layer

28

Velocity Availability Demo

demo

29

Availability Demo

Cache ClusterVelocity

Embedded in ASP.Net

Application

NorthwindApplication

ASP.net SessionCache

VELOCITYVelocity SessionStore Provider

NorthwindApplication

ASP.net SessionCache

VELOCITYVelocity SessionStore Provider

CacheViewerApplication

VELOCITY

Partitioned Cache

30

Architecture & Features

31

Velo

city

Ser

vice

Architecture

…

Server 1 Server 2 Server 3

Cache Client 2

Application/ Web Tier

Cache Tier

Configuration Store (Can be database , File share etc.)Stores Global Cache PoliciesStores Current Partitioning Information

Confi

gura

tion

Man

ager

Velo

city

Ser

vice

Velo

city

Ser

vice

Replication Substrate

Communication Substrate(Cluster Ring Management)

One of the Velocity

Service Hosts the

Configuration Manager

Velocity Client Velocity ClientVelocity Client

Users

Data manage, Object Manager

32

Velocity V1 FeaturesUsage

Key Based Caching with simple Get/Put Access and Tag based lookupOptimistic Version Based and Pessimistic Lock Based UpdatesNotification SupportHighly Available Cache with configurable number of secondariesEnterprise and Web Scale 10s – 100s of machinesIntegration with distributed management and monitoring tools

Consistency ModelsStrong/Synchronous – Synchronous communication with secondariesWeak/Asynchronous – Async communication for Higher performance

TopologiesEmbedded configuration – ASP.NetCache Service

Cache TypesPartitioned – For Activity & Resource DataReplicated – For Reference DataLocal Cache – Client cache for speeding up access

HostingIntegration with ASP.Net for session cachingHostable on IIS, Biztalk etc.

33

ASP.Net Session IntegrationSessionStoreProvider class

Plugs in to the ASP.Net Session store

Stores session state in “Velocity”

ScaleSession information accessible from all ASP.NET nodes

Enables non-sticky routing

Dynamically scale by adding new nodes

High AvailabilitySession data is backed up on to additional machines

Resilient to machine or process failures

34

Administration & Monitoring

Logging Provider model; Default ETW, support for file logs

AdministrationAdmin tool

Command Line Tool

Admin operations

Start and stop cluster or service, configure named caches ..

Integrate with Management Tools (e.g. IIS, Microsoft System Center)

Monitoring toolMonitor Cache statistics

Perfmon integration

35

Velocity Vision

In-memory DM + QPComponentized, embeddableQuery and Search

Consistent CacheTransactionsInvalidationsSynchronization

Distributed CacheDistribute Hash TableScale-outFailoverApplication Cache

CLR objects, rows, etc.ID-based access: Get/Put

Distribute Hash Table

Global Queryable,

Transactional Cache

ScalableApplication Cache

In-MemoryCache

In-Memory Data Manager

Distributed Data Manager

Distributed Data Manager,

Query Processor

Enterprise Application CacheAvailable

Application Cache

Search and LINQ queries across the distributed cache

36

Schedule

37

Schedule

June 2008 (CTP1)(TechEd 2008)

Oct 2008 (CTP2)(PDC 2008)

RTM

Distributed, partitioned cache service Regions, Tags

Local Cache Scale, Performance

ASP.Net Integration Monitoring Tools

Availability Replicated cache

Administration Tools Invalidation & Events

Multiple hosting environments

H1 CY 2009

38

Q & A

Microsoft.com Velocity sitehttp://www.microsoft.com/sql/velocity.aspx

MSDN Data Platform Development Centerhttp://msdn.microsoft.com/data

MSDN Code Galleryhttp://code.msdn.microsoft.com/velocity

Forums – Access from http://msdn.microsoft.com/data

Track Resources

www.microsoft.com/teched Tech·Talks Tech·Ed BloggersLive Simulcasts Virtual Labs

http://microsoft.com/msdn

Developer’s Kit, Licenses, and MORE!

Resources for Developers

Complete anevaluation onCommNet andenter to win!

1 Year Subscription!

42

© 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED

OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.