reimagine data governance with azure purview

42
Unified | Hybrid | Open Reimagine data governance with Azure Purview FranckMercier

Upload: others

Post on 08-Feb-2022

5 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: Reimagine data governance with Azure Purview

Unified | Hybrid | Open

Reimagine data governance with Azure Purview

FranckMercier

Page 2: Reimagine data governance with Azure Purview

Data governance is becoming increasingly

interdisciplinary

What data do I have?

Where did the data originate?

Can I trust it?

DISCOVERY

What’s my exposure to risk?

Is my usage compliant?

How do I control access & use?

What is required by regulation X?

COMPLIANCE

Data-related users

Security officers

Page 3: Reimagine data governance with Azure Purview

Elements of successful data governance

Manage growing

data landscape

Overcome

operational silos

Increase

data agility

Comply with

industry regulations

Page 4: Reimagine data governance with Azure Purview

• Fully managed, serverless, PaaS service

• Automate discovery of data in on-premises,

multicloud and SaaS sources

• Classify data at scale to specify sensitivity,

compliance, industry, business and company-

specific value

• Know where data came from and what was derived

from it with data lineage

• Deliver a curated and consistent glossary of

business terms and definitions

Reimagine data governance in the cloud

Page 5: Reimagine data governance with Azure Purview

Azure PurviewUNIFIED DATA GOVERNANCE

Data Map

• Automate and manage metadata at scale

Data Catalog

• Enable effortless discovery for data

consumers

Data Insights

• Assess data usage across your

organization

Data MapMulticloudOn-prem

Data Insights

Azure Purview

Data Catalog

SaaS

Page 6: Reimagine data governance with Azure Purview

“Data Map” = Data Assets | Lineage | Classifications

On-prem & Multicloud Operational, Analytical, SaaS

Open APIsAutomated Scanning & Classification

Azure Purview Power BI

SQL Server on-prem

Azure Synapse

Azure Data Services

M365 Compliance Center

(Apache Atlas 2.0)

Data Catalog Data Insights

Search LineageBusiness Glossary Data use reports

Azure Purview: Unified Data Governance

Publish, Discover & Curate Data

Unified Experience

Unified Platform

Page 7: Reimagine data governance with Azure Purview

Azure Purview Features at Public Preview

Azure Purview Platform

Azure Purview Studio

Azure Purview Catalog (C1)

Automated Scanning & Classification

• Dedicated per customer on shared infra• Provisioned default capacity with option to add-on capacity

Data Map

• Serverless, pay per use • Includes connectors, scanning of sources, processing into data assets, lineage capture, classification

• Search, browse, asset details • Automated meta-data and lineage extraction• Automated classification based on content inspection

• Private Endpoint • Management center

On-prem & Multi-cloud* Operational, Analytical, SaaS*

Azure Purview Data Insights (D1)

* Power BI, SQL Sever on-prem, Azure Data Services including Synapse, Cosmos DB & Storage, Non-Microsoft systems including SAP ECC, SAP S4 HANA & Teradata, Multi-cloud systems including AWS S3

• Business Glossary templates• Lineage visualization & workflows

Azure Purview Catalog included with Platform (C0)

• Catalog Insights (Asset, Scan, Glossary)• Sensitive Information Types & Labeling insights

Data Producers &

Consumers

Data Officers &

Security Officers Power BI

SQL Server on-prem

Azure Synapse

Azure Data Services

M365 Compliance Center

Open APIs

(Apache Atlas 2.0)

Page 8: Reimagine data governance with Azure Purview

The home page quick access buttons (tiles) depend on the role assigned to the user.

•For data curator, the buttons are Knowledge Center, Browse Assets, Manage Glossary and View Insights.•For data reader, the featured buttons are Knowledge Center, Browse Assets, View Glossary, and View Insights.•For data source administrator + data curator, the featured buttons are Knowledge Center, Register Data Sources, Browse Assets, and Manage Glossary.•For data source administrator + data reader, the featured buttons are Knowledge Center, Register Data Sources, Browse Assets, and View Glossary.•For data source administrator, no access to Purview Studio.

*Note: Only Owners and User Access Administrators can assign roles for Purview Studio in Azure portal.

Home Page ActivitiesHome page key activities based on assigned user roles and selected purview tiers (C0, C1 & D1)

Page 9: Reimagine data governance with Azure Purview

Power BI Integration

• Native out-of-the box connector

Page 10: Reimagine data governance with Azure Purview

Power BI Integration

• Quickly find Power BI assets:

• Workspaces

• Reports

• Datasets

• Dashboards

Page 11: Reimagine data governance with Azure Purview

Power BI Integration

Page 12: Reimagine data governance with Azure Purview

Power BI Integration

• Inherit Microsoft Information Protection (MIP)

labels after Azure Purview Scanning

• Report and Goals created on top of labeled

dataset inherit label

• Announcing Power BI inheritance of MIP

labels from Azure Synapse Analytics

(Public Preview) | Microsoft Power BI Blog

| Microsoft Power BI

Page 13: Reimagine data governance with Azure Purview

Classified as Microsoft Confidential

DEMO

Page 14: Reimagine data governance with Azure Purview

Azure PurviewStudio(Unified Experience)

Page 15: Reimagine data governance with Azure Purview

Purview StudioA single, centralized place that provides unified experience for data producers, data consumers, data & security officers

Page 16: Reimagine data governance with Azure Purview

Sources

Map your data to manage an enriched metadata map of operational and transactional data no matter where it lives

Benefits

• Automated scanning of on-prem, multicloud, SaaS data

• Discover Azure data sources, PowerBI, SQL better. Leverage turnkey integrations with Power BI, SQL (on-prem, azure, MI) and key Azure Data Services such as Azure Synapse, Cosmos DB, ADLS.

• Manage metadata and scale understanding of data with automated, fully managed, serverless metadata management capability

• Leverage Apache Atlas Open APIs to programmatically publish metadata and lineage from a wide range open-source data systems

Page 17: Reimagine data governance with Azure Purview

Sources

Automated scanning & classification of on-prem, multicloud and SaaS data

Page 18: Reimagine data governance with Azure Purview

Sources

Automated scanning & classification of on-prem, multicloud and SaaS data

Page 19: Reimagine data governance with Azure Purview

Scan Sources

Select files types for scanning. Define custom file types

Custom File Type

Page 20: Reimagine data governance with Azure Purview

Scan Sources

100+ built in classifiers, define your own custom classifiers

100 + out of box classifiers

Custom Classifiers

Page 21: Reimagine data governance with Azure Purview

Scan Sources

Run the scan one-time or on a schedule

Page 22: Reimagine data governance with Azure Purview

Purview Catalog Browse & Search(Effortless discovery of trusted & accurate data)

Page 23: Reimagine data governance with Azure Purview

Browse & Search

Discover your data based on relevance using signals derived from scanning, classification, business context

Benefits

• Empower business and technical data analysts via a catalog to find and interpret data

• Provide intelligent recommendations based on data relationships, business context, search history

• Power data scientists and engineers with business context to drive BI, Analytics, AI and ML initiatives

Page 24: Reimagine data governance with Azure Purview

Browse & Search

Search results by relevance

Benefits

• Return relevant results without writing complex queries or applying advanced filters

• Semantic search by understanding the context of every single word in search query and the intent (searching for one asset or exploration) of the user

• Support for spell check, keyword suggestions, query expansion (synonyms, semantics) and content expansion (matching the keyword with things like glossary, classification and asset name)

Page 25: Reimagine data governance with Azure Purview

Browse & Search

Filter search results by business terms, classifications, contacts

Page 26: Reimagine data governance with Azure Purview

Asset Overview

Discover operational, semantic and business information about a specific dataset

Operational Metadata

Semantic Metadata

Business Metadata

Page 27: Reimagine data governance with Azure Purview

Asset Schema

Discover technical, semantic and business information about a specific dataset

Page 28: Reimagine data governance with Azure Purview

Asset Lineage

Trace lineage of data assets across the data estate

Benefits

• Ensure data provenance with a visual representation of owners, sources, transformation, and lifecycle

• Leverage support of Apache Atlas’s open-source Lineage APIs and built-in integrations with solutions such as Azure Data Factory, Azure Data Share and Power BI

• Analyze impact of changes to data and understand dependencies visually.

• Root cause analysis of failures by inspecting dependencies upstream and determine downstream impact

Page 29: Reimagine data governance with Azure Purview

Asset Contacts

Identify experts and owners of the data asset

Page 30: Reimagine data governance with Azure Purview

Asset Related

Browse assets by hierarchy. Works for unstructured, semi-structured and unstructured data

Structured Data

Browse Hierarchy

Unstructured Data

Browse Hierarchy

Page 31: Reimagine data governance with Azure Purview

Purview Catalog Business Glossary(Search & Browse your data estate from a business lens)

Page 32: Reimagine data governance with Azure Purview

Business Glossary

Consistent and curated understanding of business terms and definitions

Benefits

• Understand business context associated with data in the organization

• Bulk Import glossary terms from existing data dictionaries easily

• Flexible business terms definition with custom attributes per business domain

• Browse & Search your data estate from a business lens

Page 33: Reimagine data governance with Azure Purview

Business Glossary

Vocabulary for business users attached to assets in the catalog

Page 34: Reimagine data governance with Azure Purview

Business Glossary

Understand business context associated with data in the organization

Page 35: Reimagine data governance with Azure Purview

Business Glossary

Bulk import business terms from existing dictionaries

Page 36: Reimagine data governance with Azure Purview

Business Glossary

Flexible business terms definition per business domain

Page 37: Reimagine data governance with Azure Purview
Page 38: Reimagine data governance with Azure Purview
Page 39: Reimagine data governance with Azure Purview

Classified as Microsoft Confidential

Purview Catalog Costs

Page 40: Reimagine data governance with Azure Purview

Understanding Azure Purview Pricing

Data Map

Data Catalogue

Scanning & Classification

Page 41: Reimagine data governance with Azure Purview

On Prem & Multi-Cloud Operational, Analytical, SaaS

Data Privacy

Data Producers & Consumers Data Engineers & SMEs Data Officers

• Business Glossary templates• Lineage visualization & workflows

Synapse

Open APIs

Power BI

AML

Sentinel

M365

SQL Server

Synapse

and more

• Catalog Insights (Asset, Scan, Glossary)• Sensitive Information Types & Labeling

insights

Data Discovery Data Use Governance

• Search, browse, asset details • Automated meta-data and lineage extraction• Automated classification based on content inspection

• Private Endpoint • Management center

Purview Data Catalog Base (C0)

* Power BI, SQL Sever on-prem, Azure Data Services including Synapse, Cosmos DB & Storage, Non-Microsoft systems including SAP ECC, SAP S4 HANA & Teradata, Multi-cloud systems including AWS S3

Purview Data Catalog (C1) Purview Data Insights (D1)

Purview Studio

Purview PlatformAutomated Scanning & Classification

• Serverless, pay per use • Includes connectors, scanning of sources, processing into

data assets, lineage capture, classification

Purview Data Map

• Dedicated per customer on shared infrastructure• Provisioned default capacity with option to add-on capacity

Azure Purview

Page 42: Reimagine data governance with Azure Purview

Total Azure Purview Cost

•Preview Offer : 4 free capacity units per month through May 31, 2021•Preview Offer : Data Map Metadata storage (3GB/CU) free