agile data management with a multi-model database · ecommerce /pricedynamically: slide: 17 31 may...

31 May 2018© MARKLOGIC CORPORATION

Justin MakeigSenior Director, Product Management

@featurebacklog

Agile Data Management with a Multi-Model Database

SLIDE: 2 31 May 2018© MARKLOGIC CORPORATION

Integrated data, Not data integration


Alternative ApproachesSILO BUSTIN’

DATA WAREHOUSES POINT-TO-POINT DATA LAKE

360º view and operational MaybePartial view, not in real-time

NoNo single source of truth

NoNot easily accessible

Agility to handle changes NoRigid schemas, ETL

MaybeO(N2) complexity

YesVery little structure

Enterprise governance YesGranular controls

MaybePoor data lineage/auditing

NoNo granularity


Heightened Urgency New competition: Non-traditional threats,

lower barriers to entry

More data: Everything is software, everything is measurable

Tougher rules: More and more comprehensive regulations


DOWNSTREAM SYSTEMS

TRANSACTIONAL APPS

OPERATIONAL APPS

Operational Data HubAGILE DATA MANAGEMENT


KEY PRINCIPLES

Enabling Agility API First

Expect and Embrace Change

Governed by Default

Deploy Anywhere


DATA AGILITY

API First Minimize up-front work by focusing on

business value and working back to data

Reduce execution risk with aggressive scoping, frequently iterations, continuous feedback

Increase returns on cumulative data

Direct analogue and enabler for Agile software development


Leading with Business ValueAPI FIRST

1. Define and Flesh Out API 2. Model and Govern

3. Load, Discover, Validate4. Map and Harmonize

1. Define and Assemble API


Return on InvestmentLEADING WITH BUSINESS VALUE

Business Value

Level of Effort

High impactPredict fraud

Flag fraud

Pricing


Run Code Close to the DataDATA LOGIC

C++ built-ins

JavaScript runtimeXQuery runtime V8

XQuery libraries JS libraries

User JavaScriptUser XQuery

Compiled engine

Scripting environment

HTTP Server External interface


New sources Messy or unexpected data Ambiguous or conflicting

definitions

New regulations and enforcement Increased threats Sharing, not hoarding

New opportunities enabled by creative reuse Get value sooner Experiment with less risk, cost


Quality and Meaning

Business Requirements

UPSTREAM DOWNSTREAM

Compliance and Governance

EVERYWHERE


ERP

/priceDynamically

Customer

Collection:/acme/customers


Hierarchical, sparse, high cardinality

Precise structure to free text

Change the data, change the schema

Standard JSON or XML, text, binary

Documents Represent Data More Naturally

DATA MODEL


ERP

Customer

Collection:/acme/customers

Document Data Model Load as-is Universal indexing

Values Full text Structure Scalar ranges Geospatial

Schema on read Organize by collections,

directories

/priceDynamically


Customer

CRM ERP eCommerce

/priceDynamically


Customer

prov:derivedFrom

prov:generatedBy

rdf:type

rdf:type

prov:wasRevisionOf

/priceDynamically

e7e9879a…

?


Semantic RelationshipsGRAPHS

Entities === Documents

Relationships as triples

- Entities related to Entities

- Entities related to Facts

- Facts related to Facts

Infer new relationships

Customer

Order

Product

Purchased

Places

Includes








Derived fromType

Type

PII

SSN








73fa4dc0…

Is Concept

Same As

SKOS


Customer

30d623ff…

,

Order Product

acme:includesacme:places

73fa4dc0…

isConcept

rdf:type

rdf:type

acme:purchased

e7e9879a…

acme:powerOfAttorney

0.

1.

rdf:type

prov:generatedBy


Governed by DefaultPUTTING THE “MS” BACK IN DBMS

Manage policy along with the data and metadata that it governs

Query that policy just like data to make enforcement model-driven

Automatically enforce policy in the database

Track lineage as data and policy change


Developer

Huh?

Domain Expert


MODEL-DRIVEN

Data, Metadata, and Policy Model important business concepts as

needed (and not before)

Manage policy along with the data and the metadata it governs

Drive business processes and configuration from queryable policy definitions

Domain Expert


Customer

e7e9879a…Secure by DesignGOVERNED BY DEFAULT

Confidentiality: Role-based access control and encryption at rest, in motion

Integrity: Transactional consistency and auditable trustworthiness

Availability: Elastic scale out and HA/DR


Deploy AnywhereDeploy AnywhereAGILE INFRASTRUCTURE

Align infrastructure costs with SLAs using elastic scaling

Avoid lock-in with flexible cloud and on premise deployment

Reduce risk with automation and componentization

https://upload.wikimedia.org/wikipedia/commons/6/67/Inside_Suite.jpg


Elastic Scaling and High AvailabilityDEPLOY ANYWHERE

DE/D



E

D

https://images.pexels.com/photos/38136/pexels-photo-38136.jpeg



E

D

E



E

D

E

D



E E E E

D D D DZo

ne 1

Zone

2


MarkLogic in the CloudCLOUD NEUTRAL

Proven in the cloud

– Private, hybrid, or public cloud

– AWS, Azure, and Google Cloud

– Deployment automation

You’ve always had the power, my dear. You just had to learn it for yourself.”

Glinda, Good Witch of the East

“


KEY PRINCIPLES

Enabling Agility API First


Governed by Default

Deploy Anywhere

https://patentimages.storage.googleapis.com/pages/US2415012-2.png


Technical breakouts for further exploration this week and beyondLearn More

Using MarkLogic Semantics for Data Integration Tuesday, 11:30 am

MarkLogic Data Hub Framework™ – A New Hope for Data Harmonization Tuesday, 3:00 pm

Deep Dive: How to Secure and Share Sensitive Information Tuesday, 2:00 pm

Best Practices for World-Class Search Tuesday, 3:00 pm

Master Data In Minutes With Smart Mastering Tuesday at 4:15

How to Govern Integrated Data (And Prove It) Wednesday, 11:00 am

MarkLogic as a Real-Time Data Hub Wednesday, 1:00 pm

MarkLogic and Microsoft Azure Accelerate Digital Transformation Wednesday, 1:00 pm

Cloud Neutral: Running MarkLogic on AWS or Azure Wednesday, 2:00 pm

Esri and MarkLogic: Location Analytics, Multi-Model Data Wednesday, 3:15 pm