přehled novinek v sql server 2016 -...

1

Přehled novinek v SQL

Server 2016

Martin Rys, BI Competency Leader

[email protected]

https://www.linkedin.com/in/martinrys

20.4.2016

mailto:[email protected]

https://www.linkedin.com/in/martinrys

BI Competency development

2

3

Trends, modern data warehousing

Current DWH/BI challenges

4

Big Data

• New data sources & types, increasing data volumes

Internet of Things (IoT)

• Sensors & devices & computers

Cloud / hybrid models

• Integration of on-premise and cloud services

Right information to all users, in relevant tools & format, at right time

• Fast data exploration, Agile BI

• Operational / Near-real time data processingand analytics

• Advanced analytics

• Self-service and Mobile BI

BICC

• New scope of work, roles, skills, tools

ODSOperationalreporting

Enterprise DWH

Semantic Models

Information Management

Relational / Structured data

Data WorkflowOrchestration

Data Transformation / Processing

DataManagement

SMP and MPP

Business Intelligence / Data Delivery

DashboardsReports

OLAP

Business Analytics

Business Analytics

Excel

Current DWH/BI

5


Enterprise DWH

Semantic Models





DataManagement

SMP and MPP


DashboardsReports

OLAP

Modern data warehousing

6

Big Data Platform

• New data sources & types, increasing data volumes

• Hadoop, NoSQL

• DWH augmentation

‒ Move from DWH centeredarchitecture to heterogeneousarchitecture


Enterprise DWH

Semantic Models


SMP and MPP


DashboardsReports

OLAP

Unstructured data StreamingBig data

Big Data PlatformData Lake

Hadoop, NoSQL




DataManagement


7

Internet of Things (IoT)

• Sensors & devices & computers

• Field Gateway, IoT Network

• Hot and Cold Path

• IoT Hubs

• Complex Event Processing (CEP)


Enterprise DWH

Semantic Models


SMP and MPP


DashboardsReports

OLAP



Hadoop, NoSQL




DataManagement

Event Processing

Event Ingestion

Complex Event Processing

Notifications

BI / Application Integration

IoT NetworkField Gateway


8

Operational / Near-real time data processing and analytics

• In-memory technologies

• In-memory OLTP

• In-memory DW

• In-memory Analytics

• In-memory Event processing


Enterprise DWH

Semantic Models


SMP and MPP


DashboardsReports

In-memory Columnar



Hadoop, NoSQL




DataManagement

Event Processing

Event Ingestion


Notifications



OLAP

In-memory technologies In-memory technologies


9

Advanced analytics

• R

• In-database data mining

• Self-service advanced analytics

Perceptual / cognitiveintelligence

• Text mining, Natural language processing, Sentiment analysis

• Image analytics

• Speech processingODSOperationalreporting

Enterprise DWH


SMP and MPP


DashboardsReports



Hadoop, NoSQL




DataManagement

Event Processing

Event Ingestion


Notifications




SemanticModels

Advanced AnalyticsPerceptual / cognitive intelligence

Machine LearningIn-database Data Mining, R

Recognition of human interaction and intentIn-memory Columnar

OLAP


10

Self-service BI

• Heterogeneous data connectivity and transformation

• Semantic layer

• Data visualization, interactive dashboards

• Report delivery, publishing

Mobile BI

• Optimized dashboards for mobile devices

• Support for Windows, iOS and Android

Real-time Dashboards

• Event monitoring


Enterprise DWH


SMP and MPP



Hadoop, NoSQL




DataManagement

Event Processing

Event Ingestion


Notifications




SemanticModels



Recognition of human interaction and intentIn-memory Columnar

OLAP


Real-time DashboardsDashboards and visualizationsReports Self-service BIMobile BI


11

BICC

• New scope of work

• New roles

• New skills

• New tools


Enterprise DWH Big Data PlatformData Lake

Event Processing

SemanticModels



Relational / Structured data Unstructured data Streaming



DataManagement

Event Ingestion


Notifications



Recognition of human interaction and intent

SMP and MPP

In-memory technologies

In-memory Columnar

In-memory technologies Hadoop, NoSQL


Real-time DashboardsDashboards and visualizationsReports Self-service BIMobile BI


Big data

OLAP

12

SQL Server 2016 Overview

SQL Server 2016

14

Excel + Power BI add-insQuery, Pivot, View, Map

SharePointPower Pivot Gallery, Power View

ExcelData Mining

Power BI Desktop Power BI Portal

Azure ML

End-to-End DW & Big Data Platform, Driving Analytics on any Data

Power BI Mobile App

Analytics Platform System(APS)

Integration with Hadoop using Polybase

15



ExcelData Mining


Azure ML

Power BI Mobile App


PolyBase provides a scalable, T-SQL compatible query processing framework for combining data from SQL Server and Hadoop / Azure blob storage

Possible Hadoop utilization and integration with DWH

16

Operational systems

External data

Relational Data

Reporting Data Mining

Data Marts

Analysis

DWH

Self-service BI Semantic models

1 2

4

3

5 5 (1) Enterprise DWH consolidates data from

Operational systems

(2) Very large / unstructured / new data and

events are stored in Hadoop

‒Data processing, exploration, analytics

‒Complex Event Processing

(3) After data processing and analysis in Hadoop

aggregated information are provided to DWH

(4) Large / cold / historical data are moved to

Hadoop, Data archiving

(5) Data stored in both DWH/BI and Hadoop are

available for BI tools

‒User friendly BI tools (including MS Excel) can be

used for analysis of data stored in Hadoop.

ETL

Data Stage

PolyBase View in SQL Server 2016

17

• Execute T-SQL queries against relational data in

SQL Server and ‘semi-structured’ data in HDFS

and/or Azure

• Leverage existing T-SQL skills and BI tools to gain

insights from different data stores

• Expand the reach of SQL Server to Hadoop(HDFS)

• Optimized for data warehousing workloads and

intended for analytical query scenarios

Query Result

PolyBase is also implemented in:

• Analytics Platform System

• Azure SQL Data Warehouse

PolyBase allows you to use Transact-SQL (T-SQL) statements to access data stored in Hadoop or Azure Blob Storage and query it in an ad-hoc fashion.

In-memory technologies

18



ExcelData Mining


Azure ML

Power BI Mobile App


Faster transactions, Faster Queries, Faster Insights

In-memory technologies overview

19

Memory-optimized OLTP engine / Hekaton In-memory Columnstore engine / xVelocity

• Row store

– Objects in-memory (memory-optimized tables, natively

compiled stored procedures)

• Integrated into SQL Server

– Tables fully transactional, durable and can be accessed using

standard T-SQL

…

Data stored as rows

• Ideal for OLTP

– Efficient operation on small set of rows

– Can significantly improve OLTP database application

performance (short-running transactions)

– Workloads where there are many inserts and updates, and

business logic in stored procedures

• Columnar store

– Data stored in columnar format / database

• Integrated into SQL Server

Data stored as columns

C1 C2 C3 C5C4

• Ideal for DW workload

– Achieves breakthrough performance for analytic queries

– Improved compression

– More data fits in memory

– Optimized for CPU utilization

– Reduced I/O: Fetch only columns needed

In-memory technologies overview

20

Faster Transactions Faster Queries Faster Insights

Up to 30x faster transaction processing

Up to 100x faster queries / stored

procedures

Improves query performance and data

compression by up to 10x

Greater performance and data

compression

In-memory OLTP In-memory DW In-memory Analytics

Memory-optimized OLTP engine /

Hekaton

• Usage

– OLTP: Highly frequent Insert/Updates,

stored procedure that does heavy

calculations

– Operational analytics: queries that

aggregate data for analysis

• Note: SSD Bufferpool Extension can also be

used to improve OLTP performance

Columnstore engine / xVelocity

• Columnstore index

– For storing and querying large tables

– Clustered columnstore index

– Nonclustered columnstore index

• Usage

– Data Warehousing

– Operational analytics

Columnstore engine / xVelocity

• In-memory analytics engine

– Provides BI semantic layer

• Usage

– SSAS Tabular / Power Pivot for

SharePoint

– Power Pivot (Excel / Power BI Desktop)

AdvancedAnalytics

21



ExcelData Mining


Azure ML

Power BI Mobile App


Enterprise-class big data analytics platform for R

22

Community Commercial

Revolution R Open

R Open

Revolution R Enterprise

R Server

Authoring tool: R Tools for Visual Studio Azure ML SDKPython / Node.js for Visual Studio

AdvancedAnalytics landscape

23

Azure Machine Learning

Business Intelligence

24


ExcelData Mining

Azure ML


Deliver right information to all users, in a way that is highly relevant and useful to each, at right time

Powerful, familiar BI tools for everyone

25

Put the power of data in the hands of every business and person on the planet

Harmonizing report types

• Paginated reports built with SQL Server Data Tools

or SQL Server Report Builder.

• Interactive reports built with Power BI Desktop.

• Mobile reports are based on Datazen technology.

• Analytical reports and charts created with Excel.

• Reporting Services is on-premises solution for BI

report delivery

• Publish Power BI Desktop Reports on-premises

• Unified Mobile BI experience

• Symmetry across on-premises and cloudRoa

dm

ap

BI Competency development

26

ADASTRA CZECH REPUBLICAdastra, s.r.o.

Karolinská 654/2, 186 00 Praha 8

Tel.: +420 271 733 303

[email protected]

www.adastra.cz

ADASTRA GROUP North America8500 Leslie St.

Markham, Ontario, L3T 7M8

Tel: +1 905 881 7946

[email protected]

Restrictions for public release and use:This document can comprise confidential information. As such it may not, without Adastra’s prior consent, be copied or transferred.

Important:All brands and names of products given in this documentation are or can be registered trademarks of their owners.© 2016 Adastra, all rights reserved.

27

Thank you!

přehled novinek v sql server 2016 -...

Documents