big data initiatives an enterprise perspective · 2015. 11. 9. · an enterprise perspective . ......

Kent Laursen, CTO, No Magic, Inc. September 15, 2014

Big Data Initiatives An Enterprise Perspective

Agenda

•  Big Data Characteristics •  Industry Trends and Uses

•  Conceptual Modeling •  Weaving the Polyglot •  Process Execution •  No Magic Roadmap

© 2014 No Magic, Inc. Exclusively for No Magic Use 2

Big Data – What is it?

•  Data sets that are too large and complex to manipulate or interrogate with standard methods or tools

•  The V4C of Big Data •  Volume – lots of it •  Variety – many kinds, both structured and unstructured •  Velocity – fast production and consumption •  Variability – changes over time •  Complexity – complicated composition and relations


Big Data – What about...?

•  Veracity •  Truth of data, pedigree, trust of source

•  Quality •  Validity, correctness, completness and integrity

•  Context •  Business: alignment with portfolios and capabilities •  Process: placement and use in operations •  System: production, transport, transformation and consumption via

automations

•  Meaning •  Understanding through conceptual description •  Weaving the polyglot


Big Data Trends

•  Hype becomes reality •  Not just Hadoop

•  More than unstructured data •  Modeling and visualization become critical

•  Conceptual/Ontology •  More sophisticated NoSQL •  Fusion with process •  Replacing legacy data management


Big Data Use – Internet of Things

•  Drivers •  Internet integration of devices •  ...Many existing and emerging use cases...

•  Data •  Sensor observations •  Commands •  Monitoring


Big Data Use - Financial

•  Drivers •  Fraud Detection •  Compliance •  Risk Management •  Integration •  Customer Relationship Management, Product Tailoring

•  Data •  Transactions •  Accounts •  Financial Instruments


Big Data Use – Bio IT

•  Drivers •  Discovery Biology, Proteomics, Genomics •  Clinical Data Analysis, Drug Research •  Disease Control, Health, Epdidemics •  Environment •  Food

•  Data •  Samples •  Instrument Output, e.g. Mass Spectrometry •  Experiments, Assays, Investigations


Big Data Use – Defense

•  Drivers •  Cybersecurity •  Intelligence Analysis •  Situational Awareness •  Alerting

•  Data •  Human Observation •  Sensor Data •  Video •  Network Monitoring Data


Conceptual Modeling Problem Statement

•  It’s hard to get a new project under way •  How does a forming team knit together the plethora of

methodologies, profiles, and plug-ins? •  How do we unify models of various data concerns across an

enterprise? •  It takes a long time to develop techniques and automation

•  Business concepts get lost in technical detail •  Many models are often necessary

•  Many profiles are at the intricate technology level (e.g., DDL, XSD, AndroMDA)

•  Too many technical choices leads to inconsistent models •  Technology concerns drag down the level of abstraction

•  It is too much work to align models, so we get disconnected silos

•  Generating systems from abstract models should be easy by now!

Conceptual Modeling Vision

•  A unifying business concept model •  Represents the concepts and defining relations of the business •  Understood and validated by business experts •  Grounded by a subset of OWL •  Can be augmented with Alf to generate an entire system

•  Used in business process models •  Connected to other models

•  Can generate a PIM from selected classes and properties •  Can be traced to any UML model, such as NIEM-UML •  Can provides a kind of “Rosetta Stone” for enterprise-level semantic

integration

•  That can generate code -- by convention •  OWL for ontologies •  DDL for databases •  XML Schema or NIEM-UML for messages

•  And keep models in sync •  Concept model changes flag other models for resolution

Concept Modeling Features

•  Abstract diagrams focus on the business •  Simpler alternative to ODM

•  Does not require full-fidelity OWL to be useful •  Crossing lines is optional •  Association class boxes are unnecessary •  «Stereotype» markup is unnecessary (for most models) •  NoTechieCamelCase for class or property names •  Uses standard UML as intended •  Encourages cleaner, hyperlinked micro-subject-area diagrams

•  Supports a glossary with plain-English statements for business expert validation

•  Generates OWL / Turtle that ontologists can augment: •  Classes •  Global properties •  Per-class property restrictions

•  Allows use of existing ontologies

Concept Modeling Features (Continued)

•  Semantically integrates multiple UML models •  Data at rest (e.g., relational DB, XML DB) •  Data in motion (e.g., XSM Schema, NIEM-UML)

•  Ties with other UML models of: •  Systems (e.g., UPDM, SysML) •  Services (e.g., SoaML)

•  Works with other standards (e.g., BPMN, SysML, UPDM)

•  Works well for MDA (i.e., forward engineering): •  Concept model plays the role of an OOA model for executable UML •  UML activities can manipulate class properties •  Concept model can generate schemas by convention rather than

markup (e.g., XML Schema, DDL, RDFS / OWL)

Concept Modeling Features (Additional)

•  Concept model provides a business-vocabulary basis for integrating master data sources (i.e., ETL)

•  A concept model is a semantic hub for multiple logical model spokes

•  Relationships between the hub and a spoke can forward generate views of data

•  Views can be used for loading data into or emulating a tuple store

•  Supports use cases such as: •  Risk analysis at a bank •  Data pull from legacy systems •  RDF data lakes integrating multiple data sources

FIBO Example - Before

FIBO Example - After

FIBO Example – Generated OWL

•  Concepts mapped to system constructs •  Provides some specification and scoping •  Multiple implemations and maintenance

Polyglot Weaving – Concept Modeling

DB External System

DB

Business Layer

Data Layer

Internal System

Conceptual Model aka Ontology

Data Layer

•  Utilize forward generation from conceptual model •  Apply frameworks and API’s •  Higher degree of reuse with less implementation and maintence

Polyglot Weaving – Concept Generation

DB External System

DB

Business Layer

Internal System

Data Layer (RDF)

•  Utilize semantic standards, OWL, RDF, SPARQL •  Both concept model and described data realized in RDF •  Polyglot data homogenized in RDF data lake

Polyglot Weaving – Semantic Data Fusion

DB External System

DB

Business Layer (SPARQL)

Internal System

Data Layer (RDF)

•  Model driven processes exercise semantic services •  Modeled logic combined with modeled data •  Migration from legacy to executable models

Polyglot Weaving – Semantic Process

DB External System

DB

Business Layer (SPARQL)

Internal System

Process Layer (BPMN)

Converge into Model Driven Ecosystem

No Magic Roadmap

•  Model the data •  Model the data configuration •  Model data fusion

•  Model and data are scalable •  Model forward generates implementions •  Model integrates with W3C and tuple Stores •  Modeled concepts traverse architecture


The Truth is in the Models

Thank You!

Questions and Dialog

Kent Laursen, CTO [email protected]


big data initiatives an enterprise perspective · 2015. 11. 9. · an enterprise perspective . ......

Documents