데이터의변화download.microsoft.com/download/e/1/3/e13a3c07-15b9-42b3... · 2018-10-16 ·...

33

Upload: others

Post on 04-Aug-2020

4 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: 데이터의변화download.microsoft.com/download/E/1/3/E13A3C07-15B9-42B3... · 2018-10-16 · reads 7591, lob logical reads 131824, lob physical reads 1, lob read-ahead reads 395064
Page 2: 데이터의변화download.microsoft.com/download/E/1/3/E13A3C07-15B9-42B3... · 2018-10-16 · reads 7591, lob logical reads 131824, lob physical reads 1, lob read-ahead reads 395064

데이터의변화

CONNECTED

DIGITAL

ANALOG

1985 1990 1995 2000 2005 2010 2015 2020

CLOUD / IoT

MOBILE

Page 3: 데이터의변화download.microsoft.com/download/E/1/3/E13A3C07-15B9-42B3... · 2018-10-16 · reads 7591, lob logical reads 131824, lob physical reads 1, lob read-ahead reads 395064

SQL Server 2016: Everything built-in

향상된 In-memory OLTP : 메모리최적화된데이터베이스크기증가, T-SQL

지원범위확장, 병렬처리및 TDE 지원

Operational Analytics: 메모리최적화및디스크기반의시스템에서도효율적인데이터분석가능

Query store: 쿼리실행이력을관리및모니터링하여보다최적의실행계획파악가능

JSON: JSON의지원으로 SQL Server로데이터를가져오거나내보내는기능가능

Temporal database: 변경된데이터이력관리가능

향상된 AlwaysOn:

• 최대 3개의동기모드의장애조치노드구성가능

• 보조서버간라운드로빈방식으로로드밸런싱

• DTC 및 SSIS 지원• 데이터베이스의상태에따른자동

장애조치

성능및확장성: Spatial, memory

management, maintenance, auto soft-

NUMA, TempDB optimization

Windows Server 2016 지원: AD 없이도AlwaysOn AG 구성가능, 향상된 OS 기능활용가능

Always Encrypted: Application의변경없이도향상된데이터보호가능

Row level security: 지정한정책(사용자기준..)에따라데이터접근수준조정가능

Dynamic data masking: 사용자권한에따라기밀데이터의 Masking 처리가능

In-memory columnstore: Batch 및Pushdown 방식의쿼리성능향상

PolyBase: 기본 T-SQL구문만으로도다양향데이터원천과의쿼리실행가능

DirectQuery: Excel, Power BI 혹은다른보고서툴등에서다양한 SQL Server

Analysis Services 모델과연결가능하며DirectQuery모드를통해직접 OLTP와연결및쿼리수행가능

향상된 SSIS: 이전버전의 SSIS Designer

지원, Power Query 지원

엔터프라이즈급Analysis Services:

복잡도높은업무(데이터)를보다쉽게모델링할수있고, In-Memory 기술을활용한 Tabular 모드의기능향상

Visual Studio 2015 기반의 SSDT: 기존의Visual Studio 2015를 이용할수있어하나의툴에서풍부한분석기반의프로젝트구현

Enhanced MDS: Excel add-in 15X faster,

more granular security roles, archival

options for transaction logs and reuse

entities across models

기본탑재된 Advanced Analytics: SQL

Server를통해 R을접할수있고, R을통한데이터분석및그에대한예측등을손쉽게할수있도록기능제공

In database analytics: 데이터이동을최소화하여고급분석기능이용가능

Scale R / Distribute R / ConnectR

Stretch Database: On-Premise의이력데이터를 Aure로자연스럽게이관하여저장비용절감가능, Always Encrypted 및RLS 유지가능

On-Premise의데이터로Power BI 이용:

Analysis Services와의연동가능. 실제데이터는 Cloud가아닌본래의위치에저장되며 SSRS에서작성한보고서를Power BI의대시보드에추가할수있는기능제공

SSIS 하이브리드시나리오: SSIS에서Azure Data Factory 연결가능

Azure로의백업: 스토리지공간 50%

절감가능, Block blobs 스토리지를이용해대용량데이터베이스백업수행가능

On-premise의 SQL Server를손쉽게Azure 마이그레이션

AlwaysOn의Azure Replica를간편하게구성: AlwaysOn을통해 On-Premise와Azure가쉽게결합되어고가용성과재해복구를동시에구성가능

개발, 운영, 관리가한자리에서가능한툴제공

SQL Server On-premise에서의경험을그대로 Azure IaaS and PaaS에서사용가능

Mobile BI: Windows, iOS 및 Android를모두지원하며온프레미스의데이터에대한통찰력을현대적이고다양한차트등으로구현

향상된 Reporting Services: 보다세련되고다양한기능을제공하며일반적인웹브라우저는물론각종모바일디바이스에도최적화된보고서작성가능

Page 4: 데이터의변화download.microsoft.com/download/E/1/3/E13A3C07-15B9-42B3... · 2018-10-16 · reads 7591, lob logical reads 131824, lob physical reads 1, lob read-ahead reads 395064
Page 5: 데이터의변화download.microsoft.com/download/E/1/3/E13A3C07-15B9-42B3... · 2018-10-16 · reads 7591, lob logical reads 131824, lob physical reads 1, lob read-ahead reads 395064

• 복잡한 시스템 구성 (2대 이상의 서버 필요

• 즉각적인 데이터분석 어려움 발생(OLTP –> DW –> Analysis –> Report)

IIS Server

데이터분석일반사용자

Page 6: 데이터의변화download.microsoft.com/download/E/1/3/E13A3C07-15B9-42B3... · 2018-10-16 · reads 7591, lob logical reads 131824, lob physical reads 1, lob read-ahead reads 395064

Management studio SQL Server 2005

Compression / Policy-based mgmt. / Programmability

PowerPivot (In-Memory) / SharePoint integration / Master data services

AlwaysOn / In-Memory ColumnStore / Data quality services / Power View / Cloud

In-memory across workloads / Performance & scale / Hybrid cloud optimized / HDInsight service / Cloud BI

Operational Analytics / Structured & unstructured data / Built-in advanced analytics / Rich visualizations for BI / Security / Hybrid Hyperscale

SQL Server 2008

SQL Server 2008 R2

SQL Server 2012

SQL Server 2014

Performance &

productivityMission critical Self-service BI Cloud-ready

Mission critical &

cloud performance

Advanced analytics &

rich visualizations

SQL Server 2016

Page 7: 데이터의변화download.microsoft.com/download/E/1/3/E13A3C07-15B9-42B3... · 2018-10-16 · reads 7591, lob logical reads 131824, lob physical reads 1, lob read-ahead reads 395064

SQL Server Engine

Transaction Log Data File Group

Ad-hoc Query & Stored

Procedures

Page 8: 데이터의변화download.microsoft.com/download/E/1/3/E13A3C07-15B9-42B3... · 2018-10-16 · reads 7591, lob logical reads 131824, lob physical reads 1, lob read-ahead reads 395064

SQL Server Engine

Memory-optimized

File Group

In-Memory OLTP

Engine: Memory-

optimized tables &

indexes

Native compiled SPs

& Schema

In-Memory OLTP

Compiler

Transaction Log Data File Group

Ad-hoc Query & Stored

Procedures

Page 9: 데이터의변화download.microsoft.com/download/E/1/3/E13A3C07-15B9-42B3... · 2018-10-16 · reads 7591, lob logical reads 131824, lob physical reads 1, lob read-ahead reads 395064

NONCLUSTERED HASH WITH (BUCKET_COUNT = 1000000)

HASH WITH (BUCKET_COUNT = 1000000)

MEMORY_OPTIMIZED = ON DURABILITY = SCHEMA_AND_DATA

Page 10: 데이터의변화download.microsoft.com/download/E/1/3/E13A3C07-15B9-42B3... · 2018-10-16 · reads 7591, lob logical reads 131824, lob physical reads 1, lob read-ahead reads 395064

항목 SQL Server 2016 SQL Server 2014

Max Size (GB)* 2048 250

ALTER TABLE*(INDEX REBUILD)

ADD/REMOVE COLUMNADD/DROP/REBUILD (BUCKET_COUNT 조정) X

ALTER PROCEDURE O X

CONSTRAINT(FK, CHECK, UNIQUE)

O X

Offline checkpoint threads

Multiple Single

Collation Full Collations Support Limited (code page 1252)

Parallelism O X

Security TDE / Always Encrypted / RLS X

LOB (n)varchar(max) / varbinary(max) X

T-SQL 상당수 제한 완화 구문 제약

Page 11: 데이터의변화download.microsoft.com/download/E/1/3/E13A3C07-15B9-42B3... · 2018-10-16 · reads 7591, lob logical reads 131824, lob physical reads 1, lob read-ahead reads 395064

ProductID OrderDate Cost

310 2014-10-01 2171.29

311 2014-10-01 415.09

312 2014-10-20 1919.03

313 2014-10-20 1295.00

ProductID OrderDate Cost

314 2015-03-01 641.22

315 2015-03-01 798.09

316 2015-03-01 1433.03

317 2016-04-06 2016.00

Page 12: 데이터의변화download.microsoft.com/download/E/1/3/E13A3C07-15B9-42B3... · 2018-10-16 · reads 7591, lob logical reads 131824, lob physical reads 1, lob read-ahead reads 395064

ProductID OrderDate Cost

310 2014-10-01 2171.29

311 2014-10-01 415.09

312 2014-10-20 1919.03

313 2014-10-20 1295.00

ProductID OrderDate Cost

314 2015-03-01 641.22

315 2015-03-01 798.09

316 2015-03-01 1433.03

317 2016-04-06 2016.00

ProductID

310

311

312

313

314

315

316

317

OrderDate

2014-10-01

--

2014-10-20

--

2015-03-01

--

--

2016-04-06

Cost

2171.29

415.09

1919.03

1295.00

641.22

798.09

1433.03

2016.00

• 메모리 및디스크모두사용

• 기존 하드웨어사용 가능

• 10~100배속도 개선

• 디자인 변경최소화

• 각각의 페이지는단일 컬럼데이터저장

• 높은 압축률 (더 많은 데이터의메모리로드 가능)

• 필요한 컬럼만을접근하여 Disk I/O의최소화

• 배치, 푸시다운 실행으로리소스 사용최적화

Page 13: 데이터의변화download.microsoft.com/download/E/1/3/E13A3C07-15B9-42B3... · 2018-10-16 · reads 7591, lob logical reads 131824, lob physical reads 1, lob read-ahead reads 395064

CLUSTERED COLUMNSTORE

Page 14: 데이터의변화download.microsoft.com/download/E/1/3/E13A3C07-15B9-42B3... · 2018-10-16 · reads 7591, lob logical reads 131824, lob physical reads 1, lob read-ahead reads 395064

CLUSTERED COLUMNSTORE

Page 15: 데이터의변화download.microsoft.com/download/E/1/3/E13A3C07-15B9-42B3... · 2018-10-16 · reads 7591, lob logical reads 131824, lob physical reads 1, lob read-ahead reads 395064

CLUSTERED COLUMNSTORE

Page 16: 데이터의변화download.microsoft.com/download/E/1/3/E13A3C07-15B9-42B3... · 2018-10-16 · reads 7591, lob logical reads 131824, lob physical reads 1, lob read-ahead reads 395064
Page 17: 데이터의변화download.microsoft.com/download/E/1/3/E13A3C07-15B9-42B3... · 2018-10-16 · reads 7591, lob logical reads 131824, lob physical reads 1, lob read-ahead reads 395064

항목 SQL Server 2016 SQL Server 2014

Clustered Columnstore Index

• 10배 이상의 압축 효과 기대 가능• B-Tree (Non-Clustered) 인덱스 추가 가능

(일정 범위 쿼리에 대한 성능 향상)• PK 및 FK 제약 조건 설정 가능

• 10배 이상의 압축 효과 기대 가능• 별도의 인덱스 추가 불가능• PK 및 FK 제약 조건 설정 불가 (Indexed View 활

용)

Non Clustered Columnstore Index

• 업데이트 가능• OLTP와 Analytics 겸용 가능• Filtered NCCI 가능• 분할된 테이블 적용 가능

• SQL Server 2012에서 처음 소개• 읽기 전용• 분할된 테이블 적용 가능

일정 범위 조건 조회• Non Clustered Index 적용으로 일정 범위에

대한 쿼리 성능 향상• 파티션 수준의 스캔 발생• 인덱스 스캔 발생

AggregatePushdown

(MIN, MAX, SUM, COUNT, AVG)

Page 18: 데이터의변화download.microsoft.com/download/E/1/3/E13A3C07-15B9-42B3... · 2018-10-16 · reads 7591, lob logical reads 131824, lob physical reads 1, lob read-ahead reads 395064
Page 19: 데이터의변화download.microsoft.com/download/E/1/3/E13A3C07-15B9-42B3... · 2018-10-16 · reads 7591, lob logical reads 131824, lob physical reads 1, lob read-ahead reads 395064
Page 20: 데이터의변화download.microsoft.com/download/E/1/3/E13A3C07-15B9-42B3... · 2018-10-16 · reads 7591, lob logical reads 131824, lob physical reads 1, lob read-ahead reads 395064

Table 'COLUMNSTORETABLE'. Scan count 36, logical reads 29976, physical reads 0, read-ahead reads 7591, lob logical reads 131824, lob physical reads 1, lob read-ahead reads 395064.

Table 'COLUMNSTORETABLE'. Segment reads 88, segment skipped 260.

Table 'Worktable'. Scan count 0, logical reads 0, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.

SQL Server Execution Times: CPU time = 1268 ms, elapsed time = 10,522 ms.

Table 'ROWSTORETABLE'. Scan count 36, logical reads 2770148, physical reads 58, read-ahead reads 499642, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.

SQL Server Execution Times: CPU time = 28545 ms, elapsed time = 151,264 ms.

* COLUMNSTORE 테이블은 약 9천만건, ROWSTORE 테이블은 약 2천5백만건의 데이터가존재하는 상황에서 수행한 결과로 동일 데이터량이 존재하는 경우로 고려하면 10.5초와 543.6초

(151초의 3.6배)로 약 52배의 쿼리 수행 속도 차이를 보임

Page 21: 데이터의변화download.microsoft.com/download/E/1/3/E13A3C07-15B9-42B3... · 2018-10-16 · reads 7591, lob logical reads 131824, lob physical reads 1, lob read-ahead reads 395064

In-memory ColumnStore

In-memory OLTP

데이터분석일반사용자

IIS Server

Page 22: 데이터의변화download.microsoft.com/download/E/1/3/E13A3C07-15B9-42B3... · 2018-10-16 · reads 7591, lob logical reads 131824, lob physical reads 1, lob read-ahead reads 395064
Page 23: 데이터의변화download.microsoft.com/download/E/1/3/E13A3C07-15B9-42B3... · 2018-10-16 · reads 7591, lob logical reads 131824, lob physical reads 1, lob read-ahead reads 395064

AlwaysOn AGFailover

ClusteringDatabaseMirroring

Log Shipping

고가용성 단위 데이터베이스 인스턴스 데이터베이스 데이터베이스

응용 프로그램연결 투명성

가용성 그룹수신기 (Listener)

가상 이름(IP Address)

Failover Partner 지정 필요

-

읽기 전용읽기 전용보조 서버

-데이터베이스

스냅숏STANDBY 모드

자동 장애조치 O O O X

보조 서버 다수 - 단일 다수

서버 구성 단일 서버 공유 자원 필요 단일 서버 단일 서버

Page 24: 데이터의변화download.microsoft.com/download/E/1/3/E13A3C07-15B9-42B3... · 2018-10-16 · reads 7591, lob logical reads 131824, lob physical reads 1, lob read-ahead reads 395064

이전 버전 (7.0)부터 사용

서버 수준 장애조치

시스템 통합 시나리오에 다수 적용

공유 스토리지 (SAN / SMB)

30초 ~ 1분동안의 장애조치 시간

서버 서비스 재시작

Passive secondary Nodes

Failover Cluster Instances

SQL Server 2012에서 소개

데이터베이스 수준 장애조치

Application의 기준에 따라 분리 가능

직접 연결된 스토리지

30초 이내의 장애조치 시간

보조 서버는 항상 온라인

Active Secondary Replicas

Availability Groups

SAN

Node1 Node2

PrimaryAutoFailover

Sync

Async

Secondary1 Secondary2

Read/Write Read

Primary

Read

Page 25: 데이터의변화download.microsoft.com/download/E/1/3/E13A3C07-15B9-42B3... · 2018-10-16 · reads 7591, lob logical reads 131824, lob physical reads 1, lob read-ahead reads 395064

이전 버전 (7.0)부터 사용

서버 수준 장애조치

시스템 통합 시나리오에 다수 적용

공유 스토리지 (SAN / SMB)

30초 ~ 1분동안의 장애조치 시간

서버 서비스 재시작

Passive secondary Nodes

Failover Cluster Instances

SQL Server 2012에서 소개

데이터베이스 수준 장애조치

Application의 기준에 따라 분리 가능

직접 연결된 스토리지

30초 이내의 장애조치 시간

보조 서버는 항상 온라인

Active Secondary Replicas

Availability Groups

Page 26: 데이터의변화download.microsoft.com/download/E/1/3/E13A3C07-15B9-42B3... · 2018-10-16 · reads 7591, lob logical reads 131824, lob physical reads 1, lob read-ahead reads 395064

Azure VMs

Mart Backup

Replica3Primary

Async

Sync

Auto Failover

Report

Replica2Replica1

• 구성마법사를 통한쉬운구성

• 최대 8개의보조 복제본

• 3개의동기 모드복제본

• 읽기 전용 라우터 목록을통한 로드 밸런싱

• 장애조치중에도읽기전용연결 지속

• Basic AG 구성 (Standard

Edition, 2 Nodes)

• AD 없이 AG 구성 가능(Windows Server 2016)

Page 27: 데이터의변화download.microsoft.com/download/E/1/3/E13A3C07-15B9-42B3... · 2018-10-16 · reads 7591, lob logical reads 131824, lob physical reads 1, lob read-ahead reads 395064

AlwaysOn AG (HA+DR)

On-Premises

동기 모드

주 서버

보조 서버

Azure VPN tunnel

BI 확장

Azure 지역1보조서버

Azure 지역2보조서버

지역간 통신

비동기 모드

Azure VPN

Page 28: 데이터의변화download.microsoft.com/download/E/1/3/E13A3C07-15B9-42B3... · 2018-10-16 · reads 7591, lob logical reads 131824, lob physical reads 1, lob read-ahead reads 395064

• 환경/구성

• 운영 상황Backup

Write

Read

SyncAsync

Variable devices

`

SyncAsync

Page 29: 데이터의변화download.microsoft.com/download/E/1/3/E13A3C07-15B9-42B3... · 2018-10-16 · reads 7591, lob logical reads 131824, lob physical reads 1, lob read-ahead reads 395064
Page 30: 데이터의변화download.microsoft.com/download/E/1/3/E13A3C07-15B9-42B3... · 2018-10-16 · reads 7591, lob logical reads 131824, lob physical reads 1, lob read-ahead reads 395064
Page 31: 데이터의변화download.microsoft.com/download/E/1/3/E13A3C07-15B9-42B3... · 2018-10-16 · reads 7591, lob logical reads 131824, lob physical reads 1, lob read-ahead reads 395064

On-premises

Cloud

• 예측불가의 요청 증가에 대한 유연한 확장• 사용한만큼만 비용 지불

APS with

HP CS300

• 성능에 가장 최적화되어 구성된 H/W&S/W 제공 (APS)• APS 대비 비용 절감이 가능하며 DW 최적화 환경 제공 (FTDW)

PRE-ENGINEERED

+

• 시스템 구현을 위한 엄격한 조건 만족• 안전하고 효율적인 데이터 관리 가능

BUILD YOUR OWN

Page 32: 데이터의변화download.microsoft.com/download/E/1/3/E13A3C07-15B9-42B3... · 2018-10-16 · reads 7591, lob logical reads 131824, lob physical reads 1, lob read-ahead reads 395064

SQL Server 2016: Everything built-in

On-premises부터 Cloud까지 모두 구성 가능

*National Institute of Standards and Technology Comprehensive Vulnerability Database updated 5/4/2015

$vs.

$$$

Page 33: 데이터의변화download.microsoft.com/download/E/1/3/E13A3C07-15B9-42B3... · 2018-10-16 · reads 7591, lob logical reads 131824, lob physical reads 1, lob read-ahead reads 395064