Microsoft SQL Server 2014: In-Memory OLTP: Memory/Storage Monitoring and Troubleshooting

Sunil Agarwal Program Managersunila@microsoft.com

DBI-B315

Session Objective(s): Demo Driven Deep Dive into

Memory and Storage: Configuration, Monitoring, Garbage Collection and Troubleshooting

Database Backup and Recovery

Key Takeaway 1You can control memory allocated to in-memory database to run in dedicated or multi-tenant environment using new DMVs and Perfmon Counters.

Key Takeaway 2Storage requirements and its management is very different compared to disk-based tables.

Session Objectives And Takeaways

Memory Monitoring and Troubleshooting

Memory Challenge

Buffer Pool

Memory Optimized

Tables

Memory Internal

Structures

Available Memory

Max S

erv

er

Mem

ory

Buffer Pool

Memory Internal

Structures

Memory Optimized

Tables

Buffer Pool

Memory Internal

Structures

Memory Optimized

Tables

Buffer Pool

Memory Internal

Structures

Memory Optimized

Tables

Memory Challenge and ResolutionMemory Pressure can lead toOther SQL workloads slow down to unacceptable performanceDML Transactions on memory-optimized tables can fail due to OOM

Possible SolutionsAllow Paging - Rejected. SQL Server does not support it.Provision and Manage Memory appropriately - Recommended

StepsEstimate Memory needed for Memory Optimized Tables - guidance providedLimit memory consumption by binding database to Resource PoolTools to monitor memory consumption for preemptive strike

Memory Optimized Tables: Garbage Cleanup

90, 150 Susan Bogota

50, ∞ Jane Prague

Timestamps NameChain ptrs City

Hash index on City

Hash index on Name

T250: lowest Active Transaction Timestamp

100, 200

John Prague

200, ∞ John Beijing

f(John)

f(Jane)

f(Beijing)

f(Prague)

Select * from <table> where name = ‘John’ Automatic Garbage Collection kicks in

Memory Optimized Tables: Garbage CleanupDesignNon-blocking, Cooperative, Efficient, Responsive, ScalableActive transactions work cooperatively and pick up parts of GC workA dedicated system thread for GC

Gen-0

Gen-15

Core-1 Core-2 Core-3 Core-4

Committed Transactions

Generations

Queues for GC work units

Committed transactions that have deleted one or more rows

Demo: Monitoring Memory and GCSunil Agarwal

Demo: OOM and Resolution

Sunil Agarwal

Demo: Shock Absorber ScenarioSunil Agarwal

Storage Configuration, Management and Troubleshooting

Storage: Data and Delta Files

Data File

Delta File

0 100

TS (ins)

RowId TableIdTS

(ins)RowId TableId

TS (ins)

RowId TableId

TS (ins)

RowIdTS

(del)TS (ins)

RowIdTS

(del)TS (ins)

RowIdTS

(del)

Check

poin

t Fi

le P

air

Row pay load

Row pay load

Row pay load

Transaction Timestamp Range

Data file contains rows inserted

within a given transaction range

Delta file contains deleted rows

within a given transaction range

Merge Operation

Memory-optimized data Filegroup

Files as of Time 600R

an

ge 1

00-

200

Ran

ge 2

00-

300

Ran

ge 3

00-

400

Ran

ge 4

00-

500

Data file with rows generated in timestamp

range

IDs of Deleted Rows (height indicates %

deleted)

Merge200-400

Deleted Files

Files Under Merge

Files as of Time 500

Memory-optimized data Filegroup

Ran

ge 1

00-

200

Ran

ge 2

00-

299

Ran

ge 3

00-

399

Ran

ge 4

00-

500

Ran

ge 5

00-

600

Ran

ge 2

00-

400

Ran

ge 2

00-

300

Ran

ge 3

00-

400

Data/Delta File State Transitions

Preallocated

Under Constructio

nActive

Merge Target

Merge Source

Required for

Backup/HA

In Transition to

TombstoneTombstone

Backup with Memory Optimized Tables

CFP State Backup in RTMPRECREATED Empty File

UNDER CONSTRUCTION Empty File

ACTIVE Only Used Bytes

MERGE SOURCE Only Used Bytes

MERGE TARGET Empty File

REQUIRED FOR BACKUP/HA Only Used Bytes

IN TRANSITION TO TOMBSTONE Empty File

TOMBSTONE Empty File

Demo: Merge Operation Life Cycle and Backups

Sunil Agarwal

In-Memory OLTP: Storage Guidelines

Configure 4x of the in-memory size of the durable tablesIn practice, we have seen update heavy workload on OLTP database the storage plateaus at < 2x

Container ConfigurationSpread IO across multiple spindlesMust meet the IOPS requirements. Typical configuration should be 3x of data generation.Spindles -

Even Numbered – created two containers on each spindleOdd Numbered – created 1 container per spindle

Demo: Storage Configuration

Sunil Agarwal

In Review: Session Objectives And TakeawaysSession Objective(s): Demo Driven Deep Dive into

Memory Configuration, Monitoring, Garbage Collection and TroubleshootingStorage Configuration, Monitoring, Garbage Collection and TroubleshootingDatabase Backup and Recovery

Key Takeaway 1You can control memory allocated to in-memory database to run in dedicated or multi-tenant environment using new DMVs and Perfmon Counters.

Key Takeaway 2Storage configuration and its management is very different compared to disk-based tables. You must account for extra storage.

Breakout Sessions (in logical order)DBI-B287 - Microsoft SQL Server 2014: In-Memory OLTP Overview

DBI-B386 - Microsoft SQL Server 2014: In-Memory OLTP for Database Developers

DBI-B385 - Microsoft SQL Server 2014: In-Memory OLTP for Database Administrators

DBI-B384 - Microsoft SQL Server 2014: In-Memory OLTP End-to-End: Preparing for Migration

DBI-B315 - Microsoft SQL Server 2014: In-Memory OLTP: Memory/Storage Monitoring and Troubleshooting

DBI-B488 - Microsoft SQL Server 2014: In-Memory OLTP Performance Troubleshooting

DBI-B313 - Microsoft SQL Server 2014: In-Memory OLTP Customer Deployments and Lessons Learned

Related content at TechEd

Track resources

Download Microsoft SQL Server 2014 http://www.trySQLSever.com

Try out Power BI for Office 365! http://www.powerbi.com

Sign up for Microsoft HDInsight today! http://microsoft.com/bigdata

Resources

Learning

Microsoft Certification & Training Resources

www.microsoft.com/learning

msdn

Resources for Developers

http://microsoft.com/msdn

TechNet

Resources for IT Professionals

http://microsoft.com/technet

Sessions on Demand

http://channel9.msdn.com/Events/TechEd

Complete an evaluation and enter to win!

Evaluate this session

Scan this QR code to evaluate this session.

© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Memory Optimized Tables: Garbage Cleanup

DesignNon-blocking, Cooperative, Efficient, Responsive, ScalableActive transactions work cooperatively and pick up parts of GC workA dedicated system thread for GC

Gen-0

Gen-15

cpu1 cpu1 cpu1 cpu1

Committed Transactions

Generations

Queues for GC work units