faster transactions & analytics with the new sql2016 in-memory technologies
TRANSCRIPT
PowerPoint Presentation
PurpleR:0 G:45 B:145Dark PurpleR:50 G:20 B:90Dark BlueR:0 G:32 B:80MagentaR:180 G:0 B:158Dark GrayR:80 G:80 B:80BlueR:0 G:120 B:215Main colorsSecondary colors (use only when necessary)Mid BlueR:0 G:24 B:143GreenR:16 G:124 B:16Light BlueR:0 G:188 B:242VMSQL2016EE\hevandvMicrosoft Worldwide Partner Conference 2016 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.16/10/16 22:121
E102Faster transactions & analytics with the new SQL2016 In-memory [email protected] Platform Solution Architect
www.henkvanderValk.com
PurpleR:0 G:45 B:145Dark PurpleR:50 G:20 B:90Dark BlueR:0 G:32 B:80MagentaR:180 G:0 B:158Dark GrayR:80 G:80 B:80BlueR:0 G:120 B:215Main colorsSecondary colors (use only when necessary)Mid BlueR:0 G:24 B:143GreenR:16 G:124 B:16Light BlueR:0 G:188 B:242Session objective:Understand the SQL Server In-Memory technology & how to benefit from it!
Key takeaways:
SQL Server 2016 In-Memory OLTP, Columnstore & In-Database R support provide serious performance gains for your transactional, data warehouse & ad-hoc Analytics query workloads!
PurpleR:0 G:45 B:145Dark PurpleR:50 G:20 B:90Dark BlueR:0 G:32 B:80MagentaR:180 G:0 B:158Dark GrayR:80 G:80 B:80BlueR:0 G:120 B:215Main colorsSecondary colors (use only when necessary)Mid BlueR:0 G:24 B:143GreenR:16 G:124 B:16Light BlueR:0 G:188 B:242
16/10/16 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.3Tech Ready 15
AgendaIntroIn-Memory OLTP & enhancementsClustered Columnstore & enhancementsDemos!In-memory OLTPData loading from flatfileMemory ManagementCopy staging data to destinationR!(Direct Query with Power BI)- Wrap-up
PurpleR:0 G:45 B:145Dark PurpleR:50 G:20 B:90Dark BlueR:0 G:32 B:80MagentaR:180 G:0 B:158Dark GrayR:80 G:80 B:80BlueR:0 G:120 B:215Main colorsSecondary colors (use only when necessary)Mid BlueR:0 G:24 B:143GreenR:16 G:124 B:16Light BlueR:0 G:188 B:242SQL Server 2016 as fraud detection scoring engine
https://blogs.technet.microsoft.com/machinelearning/2016/09/22/predictions-at-the-speed-of-data/
HTAP (Hybrid Transactional Analytical Processing)
8 socket, 192 cores16 TB RAM
PurpleR:0 G:45 B:145Dark PurpleR:50 G:20 B:90Dark BlueR:0 G:32 B:80MagentaR:180 G:0 B:158Dark GrayR:80 G:80 B:80BlueR:0 G:120 B:215Main colorsSecondary colors (use only when necessary)Mid BlueR:0 G:24 B:143GreenR:16 G:124 B:16Light BlueR:0 G:188 B:242SQL 2016: World Record Breaking Performance (TPC-H)
http://www.tpc.org/tpch/results/tpch_perf_results.asp?resulttype=noncluster
PurpleR:0 G:45 B:145Dark PurpleR:50 G:20 B:90Dark BlueR:0 G:32 B:80MagentaR:180 G:0 B:158Dark GrayR:80 G:80 B:80BlueR:0 G:120 B:215Main colorsSecondary colors (use only when necessary)Mid BlueR:0 G:24 B:143GreenR:16 G:124 B:16Light BlueR:0 G:188 B:242
Hardware
Lenovo System x3850 X66TB DDR4 4x Intel Xeon E7-4800/8800 v4 series, up to 3.2GHz, up to 1866MHz DDR4 memory access, 24 cores per processor
Lenovo System x3950 X612TB DDR4 4x Intel Xeon E7-4800/8800 v4 series, up to 3.2GHz, up to 1866MHz DDR4 memory access, 24 cores per processor
DL580 Gen9 / 6TB DDR4
HDX 24TB DDR4
Cisco UCS C460 M46TB DDR4
Dell PowerEdge R930 Rack Server6TB DDR4
Bull Bullion S16 Server
288 cores24TB
PurpleR:0 G:45 B:145Dark PurpleR:50 G:20 B:90Dark BlueR:0 G:32 B:80MagentaR:180 G:0 B:158Dark GrayR:80 G:80 B:80BlueR:0 G:120 B:215Main colorsSecondary colors (use only when necessary)Mid BlueR:0 G:24 B:143GreenR:16 G:124 B:16Light BlueR:0 G:188 B:242Microsoft Azure Cloud - SQL PAAS & VMPay for Use & Capacity on DemandAzure Virtual Machines GS5: 448 GB & up Elastic-Cloud data warehousing service: SQLDW (MPP)
PurpleR:0 G:45 B:145Dark PurpleR:50 G:20 B:90Dark BlueR:0 G:32 B:80MagentaR:180 G:0 B:158Dark GrayR:80 G:80 B:80BlueR:0 G:120 B:215Main colorsSecondary colors (use only when necessary)Mid BlueR:0 G:24 B:143GreenR:16 G:124 B:16Light BlueR:0 G:188 B:242Demos
SQL Server with 24TB RAM
& In-Memory OLTP
PurpleR:0 G:45 B:145Dark PurpleR:50 G:20 B:90Dark BlueR:0 G:32 B:80MagentaR:180 G:0 B:158Dark GrayR:80 G:80 B:80BlueR:0 G:120 B:215Main colorsSecondary colors (use only when necessary)Mid BlueR:0 G:24 B:143GreenR:16 G:124 B:16Light BlueR:0 G:188 B:242Starting SQL2016 on a server with 24 TB RAM
PurpleR:0 G:45 B:145Dark PurpleR:50 G:20 B:90Dark BlueR:0 G:32 B:80MagentaR:180 G:0 B:158Dark GrayR:80 G:80 B:80BlueR:0 G:120 B:215Main colorsSecondary colors (use only when necessary)Mid BlueR:0 G:24 B:143GreenR:16 G:124 B:16Light BlueR:0 G:188 B:242
Microsoft Worldwide Partner Conference 2016 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.16/10/16 22:1210
Question
Which data provider will be fastest for OLTP?
https://github.com/Microsoft/sql-server-samples/releases/tag/in-memory-oltp-demo-v1.0
SqlClient / OLEDB / ODBC ?
PurpleR:0 G:45 B:145Dark PurpleR:50 G:20 B:90Dark BlueR:0 G:32 B:80MagentaR:180 G:0 B:158Dark GrayR:80 G:80 B:80BlueR:0 G:120 B:215Main colorsSecondary colors (use only when necessary)Mid BlueR:0 G:24 B:143GreenR:16 G:124 B:16Light BlueR:0 G:188 B:242Post session Update : In case Step 2) Migrate SP results in an error like I encountered, please run the command : EXEC sp_changedbowner "sa -HenkMicrosoft Worldwide Partner Conference 2016 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.16/10/16 22:1211
Data loading for StagingQuestion: whats the absolute fastest way to load 7 GB of data from a single flatfile into a table?
Transport vehicles: BCP/BulkInsert/SSIS/Polybase/Powershell/External Script?
Destination:HEAP/In-memory table/CCI?
PurpleR:0 G:45 B:145Dark PurpleR:50 G:20 B:90Dark BlueR:0 G:32 B:80MagentaR:180 G:0 B:158Dark GrayR:80 G:80 B:80BlueR:0 G:120 B:215Main colorsSecondary colors (use only when necessary)Mid BlueR:0 G:24 B:143GreenR:16 G:124 B:16Light BlueR:0 G:188 B:242Demo
Data loading into
In-Memory table
PurpleR:0 G:45 B:145Dark PurpleR:50 G:20 B:90Dark BlueR:0 G:32 B:80MagentaR:180 G:0 B:158Dark GrayR:80 G:80 B:80BlueR:0 G:120 B:215Main colorsSecondary colors (use only when necessary)Mid BlueR:0 G:24 B:143GreenR:16 G:124 B:16Light BlueR:0 G:188 B:242
(demo)Bulk Insert into non-durable In-mem tableSELECT pool_id , Name , min_memory_percent , max_memory_percent , max_memory_kb/1024 AS max_memory_mb , used_memory_kb/1024 AS used_memory_mb , target_memory_kb/1024 AS target_memory_mb FROM sys.dm_resource_governor_resource_pools
BULK INSERT [LINEITEM_MO_final]FROM 'F:\TPCH\lineitem.tbl'
WITH ( FIELDTERMINATOR = '|', ROWTERMINATOR = '\n' );go2x faster than Heap!
PurpleR:0 G:45 B:145Dark PurpleR:50 G:20 B:90Dark BlueR:0 G:32 B:80MagentaR:180 G:0 B:158Dark GrayR:80 G:80 B:80BlueR:0 G:120 B:215Main colorsSecondary colors (use only when necessary)Mid BlueR:0 G:24 B:143GreenR:16 G:124 B:16Light BlueR:0 G:188 B:242Memory mgmtCREATE RESOURCE POOL [PoolHk] WITH(min_cpu_percent=0, max_cpu_percent=100, min_memory_percent=0, max_memory_percent=70, AFFINITY SCHEDULER = AUTO)
-- Dynamically change the value of MAX_MEMORY_PERCENT ALTER RESOURCE POOL PoolHk WITH (MAX_MEMORY_PERCENT = 90 ) -- RECONFIGURE enables resource governor ALTER RESOURCE GOVERNOR RECONFIGURE ALTER RESOURCE GOVERNOR RESET STATISTICS
--- BIND the Database to the Poolexec sys.sp_xtp_bind_db_resource_pool N'TicketReservations', N'PoolHk';-- Check memory assignments:SELECT pool_id , Name , min_memory_percent , max_memory_percent , max_memory_kb/1024 AS max_memory_mb , used_memory_kb/1024 AS used_memory_mb , target_memory_kb/1024 AS target_memory_mb FROM sys.dm_resource_governor_resource_pools -- take DB offline:ALTER DATABASE [TicketReservations] SET OFFLINE WITH ROLLBACK IMMEDIATE
-- Bring the Database back Online:ALTER DATABASE [TicketReservations] SET ONLINE
-- Deleting the Resource Pool:-- sys.sp_xtp_unbind_db_resource_pool N'TicketReservations'-- DROP RESOURCE POOL [PoolHk]
A binding has been created. Take database 'TicketReservations' offline and then bring it back online to begin using resource pool 'PoolHk'.
PurpleR:0 G:45 B:145Dark PurpleR:50 G:20 B:90Dark BlueR:0 G:32 B:80MagentaR:180 G:0 B:158Dark GrayR:80 G:80 B:80BlueR:0 G:120 B:215Main colorsSecondary colors (use only when necessary)Mid BlueR:0 G:24 B:143GreenR:16 G:124 B:16Light BlueR:0 G:188 B:242Extended Event tracing for in-memory OLTP:Do not use SQL Profiler for in-memory profiling
CREATE EVENT SESSION [NCsp_completed] ON SERVER ADD EVENT sqlserver.sp_statement_completed( ACTION(sqlserver.session_id,sqlserver.session_nt_username,sqlserver.sql_text,sqlserver.tsql_stack) WHERE ([sqlserver].[database_id]=(7)))
ADD TARGET package0.ring_bufferWITH ( MAX_DISPATCH_LATENCY = 1 SECONDS )GO
ALTER EVENT SESSION [NCsp_completed] ON SERVER STATE = START/STOP
DROP EVENT SESSION [NCSP_completed] ON SERVER
PurpleR:0 G:45 B:145Dark PurpleR:50 G:20 B:90Dark BlueR:0 G:32 B:80MagentaR:180 G:0 B:158Dark GrayR:80 G:80 B:80BlueR:0 G:120 B:215Main colorsSecondary colors (use only when necessary)Mid BlueR:0 G:24 B:143GreenR:16 G:124 B:16Light BlueR:0 G:188 B:242SQL 2016 Columnstore
PurpleR:0 G:45 B:145Dark PurpleR:50 G:20 B:90Dark BlueR:0 G:32 B:80MagentaR:180 G:0 B:158Dark GrayR:80 G:80 B:80BlueR:0 G:120 B:215Main colorsSecondary colors (use only when necessary)Mid BlueR:0 G:24 B:143GreenR:16 G:124 B:16Light BlueR:0 G:188 B:242
Data Loading into Columnstore: Bulk ImportKey PointsEach Bulk Import thread takes X lock on the RG. If batchsize < 100k, the rows are inserted into a delta RG otherwise a new compressed RG is generatedMinimal Logging (only when imported into compressed rowgroup)Dont specify TABLOCK when importing unlike rowstore where it is required
Compressed RGsPage compressed Delta RGsBulk Import (batchsize < 102400 )
X rowgroup lockBulk Import (batchsize >= 102400 )
PurpleR:0 G:45 B:145Dark PurpleR:50 G:20 B:90Dark BlueR:0 G:32 B:80MagentaR:180 G:0 B:158Dark GrayR:80 G:80 B:80BlueR:0 G:120 B:215Main colorsSecondary colors (use only when necessary)Mid BlueR:0 G:24 B:143GreenR:16 G:124 B:16Light BlueR:0 G:188 B:242Bulk Insert into CCI
BULK INSERT [TicketReservations].[dbo].[LINEITEM_CCI]FROM 'F:\TPCH\lineitem.tbl'WITH ( --TABLOCK, FIELDTERMINATOR = '|',ROWTERMINATOR = '\n' ,BATCHSIZE = 204800
);8min 9sec123K Rows/sec!
PurpleR:0 G:45 B:145Dark PurpleR:50 G:20 B:90Dark BlueR:0 G:32 B:80MagentaR:180 G:0 B:158Dark GrayR:80 G:80 B:80BlueR:0 G:120 B:215Main colorsSecondary colors (use only when necessary)Mid BlueR:0 G:24 B:143GreenR:16 G:124 B:16Light BlueR:0 G:188 B:242SSIS load into a Clustered Columnstore table7.2GB in 9min 51sec ==100K Rows/sec
Default loads into Delta Store!
PurpleR:0 G:45 B:145Dark PurpleR:50 G:20 B:90Dark BlueR:0 G:32 B:80MagentaR:180 G:0 B:158Dark GrayR:80 G:80 B:80BlueR:0 G:120 B:215Main colorsSecondary colors (use only when necessary)Mid BlueR:0 G:24 B:143GreenR:16 G:124 B:16Light BlueR:0 G:188 B:242SSIS 2016
Set the DefaultBuffer to 10x larger , like 104857600 for direct load into RowstoreNEW! Leverage SSIS 2016 AutoAdjustbufferSize = True (For wide tables)
PurpleR:0 G:45 B:145Dark PurpleR:50 G:20 B:90Dark BlueR:0 G:32 B:80MagentaR:180 G:0 B:158Dark GrayR:80 G:80 B:80BlueR:0 G:120 B:215Main colorsSecondary colors (use only when necessary)Mid BlueR:0 G:24 B:143GreenR:16 G:124 B:16Light BlueR:0 G:188 B:242(max buffer size is 100 MB per session)
Microsoft Worldwide Partner Conference 2016 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.16/10/16 22:1221
SSIS ADO.NetDefaults into deltastore
Description of the persistent state of the row group:
INVISIBLE A hidden compressed segment in the process of being built from data in a delta store. Read actions will use the delta store until the invisible compressed segment is completed. Then the new segment is made visible, and the source delta store is removed.
OPEN A read/write row group that is accepting new records. An open row group is still in rowstore format and has not been compressed to columnstore format.
CLOSED A row group that has been filled, but not yet compressed by the tuple mover process.
COMPRESSED A row group that has filled and compressed.
PurpleR:0 G:45 B:145Dark PurpleR:50 G:20 B:90Dark BlueR:0 G:32 B:80MagentaR:180 G:0 B:158Dark GrayR:80 G:80 B:80BlueR:0 G:120 B:215Main colorsSecondary colors (use only when necessary)Mid BlueR:0 G:24 B:143GreenR:16 G:124 B:16Light BlueR:0 G:188 B:242BCP bcp [TicketReservations].[dbo].[LINEITEM_CCI] in F:\TPCH\lineitem.tbl -S VMSQL2016EE\SQL2016RTM -c -T -t"|"
Default is okay - into CCI (ignore the 1000 rows message on screen)Adding a larger batchsize:-b990000 gave best throughput of 102K Rows/sec
PurpleR:0 G:45 B:145Dark PurpleR:50 G:20 B:90Dark BlueR:0 G:32 B:80MagentaR:180 G:0 B:158Dark GrayR:80 G:80 B:80BlueR:0 G:120 B:215Main colorsSecondary colors (use only when necessary)Mid BlueR:0 G:24 B:143GreenR:16 G:124 B:16Light BlueR:0 G:188 B:242R
INSERT INTO [dbo].[LINEITEM_MO_final] EXEC sp_execute_external_script @language = N'R', @script = N'OutputDataSet 1 in production1M requests/sec in testing
PurpleR:0 G:45 B:145Dark PurpleR:50 G:20 B:90Dark BlueR:0 G:32 B:80MagentaR:180 G:0 B:158Dark GrayR:80 G:80 B:80BlueR:0 G:120 B:215Main colorsSecondary colors (use only when necessary)Mid BlueR:0 G:24 B:143GreenR:16 G:124 B:16Light BlueR:0 G:188 B:242In-Memory OLTP featuresMemory-Optimized TablesData storage in memory-optimized structuresCompiled into DLLs for efficient data accessMemory-Optimized Table TypesMemory-optimized table variables live in memory, in user DB (no tempdb, no IO)Can be used for TVPs and table variables in all stored procs (not just native)Natively Compiled Stored ProceduresOptimize performance of OLTP-style operationsNot suitable for reporting/analytics style queriesNatively Compiled Scalar User-Defined FunctionsDo not require memory-optimized tables if there is no data access
PurpleR:0 G:45 B:145Dark PurpleR:50 G:20 B:90Dark BlueR:0 G:32 B:80MagentaR:180 G:0 B:158Dark GrayR:80 G:80 B:80BlueR:0 G:120 B:215Main colorsSecondary colors (use only when necessary)Mid BlueR:0 G:24 B:143GreenR:16 G:124 B:16Light BlueR:0 G:188 B:242
TechReady 23 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.16/10/16 22:1234
Create Table DDLCREATE TABLE [Customer]( [CustomerID] INT NOT NULL PRIMARY KEY NONCLUSTERED, [Name] NVARCHAR(250) NOT NULL, [CustomerSince] DATETIME2 NULL,INDEX [ICustomerSince] NONCLUSTERED (CustomerID, CustomerSince)
)WITH (MEMORY_OPTIMIZED = ON, DURABILITY = SCHEMA_AND_DATA);
This table is durable (default).Non-durable tables:DURABILITY=SCHEMA_ONLY
PurpleR:0 G:45 B:145Dark PurpleR:50 G:20 B:90Dark BlueR:0 G:32 B:80MagentaR:180 G:0 B:158Dark GrayR:80 G:80 B:80BlueR:0 G:120 B:215Main colorsSecondary colors (use only when necessary)Mid BlueR:0 G:24 B:143GreenR:16 G:124 B:16Light BlueR:0 G:188 B:242
35
Create Table Type DDLCREATE TYPE [Sales].[SalesOrderDetailType_inmem] AS TABLE([OrderQty] [smallint] NOT NULL,[ProductID] [int] NOT NULL,[SpecialOfferID] [int] NOT NULL,
INDEX [IX_ProductID] NONCLUSTERED HASH ([ProductID]) WITH ( BUCKET_COUNT = 8)
) WITH ( MEMORY_OPTIMIZED = ON )
PurpleR:0 G:45 B:145Dark PurpleR:50 G:20 B:90Dark BlueR:0 G:32 B:80MagentaR:180 G:0 B:158Dark GrayR:80 G:80 B:80BlueR:0 G:120 B:215Main colorsSecondary colors (use only when necessary)Mid BlueR:0 G:24 B:143GreenR:16 G:124 B:16Light BlueR:0 G:188 B:242Create Procedure DDLCREATE PROCEDURE [dbo].[InsertOrder] @id INT, @date DATETIME2 WITH NATIVE_COMPILATION, SCHEMABINDING
AS BEGIN ATOMIC WITH (TRANSACTION ISOLATION LEVEL = SNAPSHOT, LANGUAGE = NDutch')
-- insert T-SQL hereEND
PurpleR:0 G:45 B:145Dark PurpleR:50 G:20 B:90Dark BlueR:0 G:32 B:80MagentaR:180 G:0 B:158Dark GrayR:80 G:80 B:80BlueR:0 G:120 B:215Main colorsSecondary colors (use only when necessary)Mid BlueR:0 G:24 B:143GreenR:16 G:124 B:16Light BlueR:0 G:188 B:242
37
Create Function DDLCREATE FUNCTION [dbo].[ufnGetAccountingEndDate_native]()RETURNS [datetime] WITH NATIVE_COMPILATION, SCHEMABINDINGAS BEGIN ATOMIC WITH (TRANSACTION ISOLATION LEVEL=SNAPSHOT, LANGUAGE=N'us_english')
RETURN DATEADD(millisecond, -2, CONVERT(datetime, '20160701', 112));
END
PurpleR:0 G:45 B:145Dark PurpleR:50 G:20 B:90Dark BlueR:0 G:32 B:80MagentaR:180 G:0 B:158Dark GrayR:80 G:80 B:80BlueR:0 G:120 B:215Main colorsSecondary colors (use only when necessary)Mid BlueR:0 G:24 B:143GreenR:16 G:124 B:16Light BlueR:0 G:188 B:242
38
Durability optionsDurability levelConfigurationPros/ConsScenariosFull durabilityDefaultDURABILITY=SCHEMA_AND_DATAPro:Every committed change is guaranteed to survive failureCon: Latency impact: every commit requires log IODefault: most scenarios need full durabilityDelayed durabilityTransaction commit timeCOMMIT WITH (DELAYED_DURABILITY=ON)Atomic block of native procedureBEGIN ATOMIC WITH (DELAYED_DURABILITY=ON, )Database levelALTER DATABASE CURRENT SET DELAYED_DURABILITY=FORCEDPro:Low latency due to no log IO in transaction execution pathEfficient log IO due to batchingCon:Limited data loss on failure (usually ~60K or ~1ms worth)Low latency requirements
Can accept some data lossORCopy of recent data exists elsewhere in case of failure
AlwaysOn auto-failover (sync replicas) with low latencyNon-durable tablesTable creationDURABILITY=SCHEMA_ONLYPro:No IO at allCon:Lose data on failureTransient data such as session stateCachingETL (staging tables)
PurpleR:0 G:45 B:145Dark PurpleR:50 G:20 B:90Dark BlueR:0 G:32 B:80MagentaR:180 G:0 B:158Dark GrayR:80 G:80 B:80BlueR:0 G:120 B:215Main colorsSecondary colors (use only when necessary)Mid BlueR:0 G:24 B:143GreenR:16 G:124 B:16Light BlueR:0 G:188 B:242Allow 32-bit operating systems to access large amounts of memory!Introduced in SQL2000 Adress Windowing Extensions (AWE)
sp_configure 'awe enabled', 1Reconfigurego
PurpleR:0 G:45 B:145Dark PurpleR:50 G:20 B:90Dark BlueR:0 G:32 B:80MagentaR:180 G:0 B:158Dark GrayR:80 G:80 B:80BlueR:0 G:120 B:215Main colorsSecondary colors (use only when necessary)Mid BlueR:0 G:24 B:143GreenR:16 G:124 B:16Light BlueR:0 G:188 B:242
Analysis Services In-MemoryIn-Memory Analytics ColumnStore (NCCI)PowerPivot In-Memory for Excel
The Journey to Complete In-Memory
PowerPivot In-Memory for Excel
2012SQL Server
2008 R2SQL Server
+In-Memory Analytics ColumnStore (CCI)In-Memory OLTPAnalysis Services In-MemoryPowerPivot In-Memory for Excel
2014SQL Server41
+Real Time Operational AnalyticsIn-Memory OLTPAnalysis Services In-Memory (10x faster)PowerPivot In-Memory for Excel
Azure SQL DB and Azure SQL DW
2016SQL ServerAPS- introductionIn-Memory Analytics ColumnStore (CCI)
PurpleR:0 G:45 B:145Dark PurpleR:50 G:20 B:90Dark BlueR:0 G:32 B:80MagentaR:180 G:0 B:158Dark GrayR:80 G:80 B:80BlueR:0 G:120 B:215Main colorsSecondary colors (use only when necessary)Mid BlueR:0 G:24 B:143GreenR:16 G:124 B:16Light BlueR:0 G:188 B:242Lets start off by taking a look at how in-memory technologies have evolved in SQL Server. This may be a surprise to many of you, but our in-memory journey actually started way back in SQL Server 2008 R2, when our engineering team made a key design decision to build in-memory technology into the core data platform, rather than acquire and stitch in an in-memory solution to run in parallel to the core database.
We believe our customers want to utilize other key capabilities that SQL Server and the broader Microsoft Data Platform have to offer in conjunction with in-memory. They dont want to use a different tool set for in-memory database than they do for disk database, they still wan to use T-SQL and SQL Server management when enabling in-memory. This is the unique design approach we took back when we first started improving analytics by building in-memory into PowerPivot for billions for rows of data analysis in Excel.
Then in SQL Server 2012, we expanded our in-memory footprint with the same built-in approach by adding in-memory to Analysis Services so IT could build data models much faster, and introduced an in-memory column store that could improve query speeds by 100x.
With SQL Server 2014, we are covering the final workload by introducing an in-memory OLTP solutionor in-memory rowstoreto significantly speed transactional performance. We also enhanced the in-memory columnstore with faster performance and significantly higher data compression so memory utilization can be optimized.
SQL Server 2014 is our third release of in-memory solutions across the data platform, and in a single sku, no options or add ons.
2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.16/10/1641
Performance gains using Fast PathOpportunistic evaluation of aggregation in storage layerEfficient Aggregation on compressed/encoded data in cache-friendly execution (column at a time) Leverages SIMD Datatype = 100kIX rowgroup lockRows < 100kKey Points:Similar to Bulk Import but with single BATCHResidual rows < 100k are inserted into delta storeSQL 2016: Parallel Insert into Columnstore index
Delete bitmapInsert into select * from
Using tablock!
PurpleR:0 G:45 B:145Dark PurpleR:50 G:20 B:90Dark BlueR:0 G:32 B:80MagentaR:180 G:0 B:158Dark GrayR:80 G:80 B:80BlueR:0 G:120 B:215Main colorsSecondary colors (use only when necessary)Mid BlueR:0 G:24 B:143GreenR:16 G:124 B:16Light BlueR:0 G:188 B:242
Thank you!
Hope to See you again @ SQL Saturday!Oct 15 2016 - Universiteit Utrecht!
http://www.sqlsaturday.com/551/eventhome.aspxwww.henkvandervalk.com
2016 Microsoft Corporation. All rights reserved.
PurpleR:0 G:45 B:145Dark PurpleR:50 G:20 B:90Dark BlueR:0 G:32 B:80MagentaR:180 G:0 B:158Dark GrayR:80 G:80 B:80BlueR:0 G:120 B:215Main colorsSecondary colors (use only when necessary)Mid BlueR:0 G:24 B:143GreenR:16 G:124 B:16Light BlueR:0 G:188 B:242