solving industrial data integration with machine intelligence

1 Bit Stew Systems Inc. Confidential and Proprietary

Solving Industrial Data Integration with Machine Intelligence

By: Mike Varney

Executive Director, Product Management & Strategic Initiatives at Bit Stew


The Problem with Big Data

With 50 billion devices connected to the Internet by 2020 rapid data proliferation will continue to choke the progress of data analytics projects.

Analytics is only the tip of the iceberg. Data integration, data quality, and challenges in extracting value lurk beneath the surface and are amplified by the growth of Industrial data from IIoT.

Analytics are the tip of the iceberg

Data cleansing and quality challenges lurk beneath the surface


Two Different Worlds, One Common Problem

Operational Technology (OT)

Information Technology (IT)

Industrial IoT

• Asset Performance • Real-Time Status Monitoring• Diagnostics & Service• Predictive Maintenance• Software Updates

• Data Integration• Data Cleansing • Systems Management• Analytics• Value-Added Services

0101010010101

0101


Hidden Cost of Integration

• Industrial data is not easy• Complex data, variable size, frequency, and format • Disparate sources • No view of data relationships across the enterprise • Often rely on traditional ETL or BI tools• Large integration teams costing thousands of dollars


Complex Environments Lead to Complex Data

Why is Industrial Data so complicated?

Variety of source types Messy Data Complex Data Relations

Different Sizes Massive Volume Varied Frequency


Traditional ETL Compounds the Problem

This model does not work for large volumes of complex OT and IT data

Extracts source data without contextual understanding

Then a team of integrators, architects, and data scientists apply transformations

After which, finally it can be provided to business and operational teams


Why MIx Core?

Small teams can solve BIG industrial problems with MIx Core™

• No longer need large and costly integration teams

• IoT endpoints are being added at an accelerated pace that is unmanageable by human computing

• Machine Intelligence solves the compounding data problem in Industrial environments


Using Machine Intelligence to Automate Data Integration

• Comprehensive Semantic Model

• Automatic sensing of data and mapping to Semantic Model

• Dynamic and adaptive to source data

• Examine, transform and clean data

MachineIntelligence

Create Semantic ModelSiloed Data Sources

Intelligent Integration to Target Model

• Integrate with APIs, services, files, streams and network traffic

• Embedded guaranteed messaging

• Interactive UI for viewing and editing


Step 1: Create Feature Vector

– Source data is read and sent to the Hierarchical Feature Extractor

– The Hierarchical Feature Extractor produces Feature Vectors

– The Hierarchical Feature Extractor can be customized for domain specific feature extraction and machine learning.

– Feature vectors are used to fingerprint and classify the input data.

Step 2: Store denormalized data into the Data Index

– The Feature Vectors are stored into the Data Index

– Data is denormalized and fed into the Relationship Association

– A Relationship Matrix is created and stored into the Data Index

How it Works?


How it Works?

Step 3: Create Semantic Model

– The Feature Vectors are sent to the Field Classifier

– The Field Classifier uses supervised and unsupervised machine learning algorithms to relate and map data

– Each algorithm gets a vote to determine the best model

– The related and mapped data is sent to the Modeler to create a Semantic Model

Step 4: Publish the Semantic Model to Target Systems

– Model is published and available for use in the platform

– The semantic model may be exported to 3rd party systems for further analysis

– Changes to published model are tracked over time for the system to learn how relationships in the data change over time


Learn more about Bit StewVisit www.info.bitstew.com

Follow us on

Download the whitepaper: http://info.bitstew.com/whitepaper-mike-machine-intelligence


Mike Varney spent over 20 years in the US Navy, where his experience included commanding the most advanced nuclear-powered submarines in complex operations around the globe, leading a special operations team in reconstruction efforts in Afghanistan, and directing a Naval Operations Centre. He has also served as a Strategic Advisor for the US Department of Defense, a Senior Evaluation Officer at nuclear power plants, and an advisor to companies providing smart grid technologies to the energy industry.

Mike holds Bachelor of Science degrees in Nuclear and Marine Engineering as well as Master of Science degrees in Engineering Management and National Security Strategy. Today, Mike is the Executive Director, Product Management & Strategic Initiatives, where he leads the strategy for Bit Stew Systems MIx Core platform, MIx Developer Network and Bit Stew University.

About the Author