industrial iot solution architecture design – from …...azure stream analytics has built-in,...

Industrial IoT Solution Architecture Design –From Connectivity to Data

Cheryl Hsu

Program Manager

Strategic Engagement & Industrial IoT, Microsoft

OPERATIONS

PEOPLE

PRODUCTSCUSTOMERS

INTELLIGENCE

DATA

IoT Enables a Digital Feedback LoopThe benefits are profound

IoT enables a “digital feedback loop”

that connects

▪ Customers

▪ Operations

▪ Products/Assets

▪ Employees

Our vision is to help businesses take advantage

of the digital feedback loop

INSIGHTS

ACTIONSTHINGS

Digital Feedback Loop

A realtime

connection

enables new

breakthrough

levels of insights

that in turn drive

informed actions

With IoT Access devices remotely

to diagnose and resolve

issues

Complete corrections

within hours, including

rerouting processes and

reconfiguring machines

Access comprehensive data

immediately to perform

root-cause analysis

1 2 3

With

fragmented

solutions

Maintain technicians onsiteto determine and resolve issues

Take days or weeks to reroute and reconfigure devices

Search for data needed for root-cause analysis

1 2 3

Respond and recover quickly

OR OR

With

fragmented

solutions

Solve storage on your own using capacity planning, capital purchases and on-going maintenance

Connect new devices later after customizations and integration efforts are complete

Take weeks or months to modify and extend systems with custom connections

1 2 3

Expand, change and scale easily

+ +

With IoT Exploit cloud solutions to

scale instantly and pay for

only what you need

Connect new devices now

with little or no

configuration required

Add to and extend systems

faster by building on the

extensible architecture

1 2 3

INSIGHTS

ACTIONSTHINGS

Updating devices

Provisioning devices

Device updates

Data storage

Cold path analytics

Warm path analytics

Hot path analytics

On device analytics

Securing data

Business process integration

Solution scale

High availability

Disaster recovery

Transport protocols

Cost management

Operations monitoring

Device lifecycle

Data ownership

Data visualization

Cloud-to-devicecommands

< ---- End-to-End Security ---- >

Industry and government compliance

Enterprise integration

Device recoveryInternationalization

HW certificationManufacturing scale

CI/CD

Drivers

Device commercialization

Enabling the Digital Feedback Loop used to be challenging

Before working on Architecture Design

Understand your business needs and challenges

Define value

proposition

Understand

organizational

impact

Assess

organizational

capability

Secure

business

stakeholder

buy-in

Input to

Architectural

Design

Getting Started

IoT Reference Architectures

IoT Reference Architecture

The latest Azure IoT cloud native

recommended architecture and latest

technology implementation

recommendations.

Provides:

▪ Overview of the IoT space

▪ Recommended subsystem factoring for solutions

▪ Prescriptive technology recommendations per subsystem

▪ Proven production ready architecture

▪ Proven technology implementation choices

▪ Recommendations for scaling systems

▪ Reference architecture implementations such as Remote

Monitoring and Connected Factory.

https://aka.ms/iotrefarchitecture

ActionsThings Insights

• Every organization has unique skills and experience

• Every IoT application has unique needs and considerations

Core Subsystems

UI & Reporting

Tools

Storage

Provision and send data from device to cloud

Stream processing and rules

evaluation over data

Visualize data and learnings

Store dataIntegrate with Business processes

Cloud Gateway (IoT Hub)

Stream Processing

Business IntegrationDevice

Management

IoT DevicesIoT DevicesIoT Devices

Azure IoT reference architecture

All Subsystems – Lambda Architecture

UI & Reporting

Tools

Warm PathStore

Stream processing and rules evaluation over data

Visualize data and learnings

Integrate with Business processes

Cloud Gateway (IoT Hub)

Stream Processing

Business Integration

Device Management

IoT DevicesIoT DevicesIoT Devices

Machine Learning

Cold PathStore

User Management

IoT DevicesIoT DevicesIoT Edge Devices

Bulk Device Provisioning

Data Transformation

Store data

Fast Path – Real Time Processing

Slow Path – Batch Processing

All Subsystems and Cross-Cutting needs

UI & Reporting

Tools

Warm PathStore

2.1 Stream processing and rules evaluation is done for device telemetry records

and events.

Visualize data and learnings

Cloud Gateway

Stream Processing

Business Integration

Device Management

IoT DevicesIoT DevicesIoT Devices

Machine Learning

Data Transformation

Cold PathStore

User Management

IoT DevicesIoT DevicesIoT Edge Devices

Bulk Device Provisioning

Deployment

High Availability and Disaster Recovery

Security

Store data

1. Devices send telemetry records or events to the cloud gateway.

2.1 Device telemetry data is transformed if needed.

2.2 Device data telemetry is stored.

3. Business Process integration (email, CRM,

etc.) is executed.

4. Device information is visualized and shown in UI.

4. ML processing is done for data.

The industry’s most agile, comprehensive, and secure portfolio

Azure IoT (PaaS)

Partner repeatable solutions

Azure IoT Solution Accelerators

Data and Analytics

Azure Time Series

Insights

Azure

Machine Learning

Cosmos DB

Azure Stream

Analytics

Azure Data Lake

Azure Data Lake

Analytics

Azure HD Insight

Visualization and Integration

Azure Logic Apps

Notification Hubs

Azure Websites

Microsoft Flow

Microsoft

Power BI

Azure Monitor

Azure Active Directory

Device support

Azure IoT

Device SDK

Azure IoT

certified devices

Security Program for

Azure IoT

Windows 10 IoT

IoT

Edge

Azure IoT Hub

Azure IoT Edge

IoT Hub Device

Provisioning Service

Microsoft Dynamics

Connected Field Service

Azure IoT Central

IoT SaaS

Solutions (PaaS)

Technologies (PaaS)

Solutions (SaaS)

SaaS, PaaS, and IaaS Guidance

Azure IoT solution

accelerators

Predictive Maintenance

Connected FactoryRemote Monitoring

Device Simulation

End-to-end implementation

Completely customizable

Open-source microservices based architecture

Device connectivity and management

Dashboards, visualization, and insights

Workflow automation and integration

Command and control

Preconfigured solutions

Accelerate time to value

Start quickly for

common IoT scenarios

Get started in minutes

Modify existing rules and alerts

Add your devices and begin tailor to your needs

Finish with your

IoT application

Fine-tuned to specific assets and processes

Highly visual for your real-time operational data

Integrate with back-end systems

Microsoft Azure IoT Solution Accelerators

https://www.azureiotsolutions.com/Accelerators

Azure IoT Connected Factory

Azure IoT Gateway SDK

On-Premise: Device Connectivity Cloud: Data Ingestion & Processing, Command & Control Cloud: Presentation

Indu

stri

al D

evic

es(O

PC-U

A S

erve

rs)

Hot Path Analytics

Azure Stream Analytics, Azure Storm, …

Azure IoT Hub

OPC Clients, Servers, ERP Portals,

OPC Graph Database and OPC UA .NET Standard Stack

JSON/AMQPUA Binary

Other Devices

OPC UA Client Module

IoT Proxy Module

UA Binary/AMQP

UA Binary

JSON/AMQP

Any

Firewall

Cold Path Analytics & Storage

Azure HD Insight, Azure Storage, SQL, DocDB, …

Presentation & Business Connections

Websites, Mobile Services

Dynamics, BizTalk Services, Notification Hubs

Microsoft

Dynamics

OPC UA Integration into Azure IoT

Industrial IoT

Critical needs in Industrial IoT

• Interoperability

• Scalability

• Precision

• Programmability

• Low latency

Connecting your factory with IoT

Global PLC & Industrial network market share

Global PLC market share as of 2017, by manufacturer Industrial network market shares 2016 according to HMS

https://www.statista.com/statistics/897201/global-plc-market-share-by-manufacturer/ https://iebmedia.com/index.php?id=11451&parentid=74&themeid=255&hft=93&showdet

ail=true&bb=1

Siemens

Mitsubishi

Omron

Schneider

Rockwell

B&R

GE

ABB

Others

Connectivity Complexity

Machinery Equipment

• CNC

• Injection Molding

• More…

Connectivity

• Ethernet capable?

• Serial standard?

• Others…

PLC

• Brand

• Driver

• OPC UA supported?

P

Manufacturing

Communication Layer

Communication Layer

Communication Layer

Enterprise & plant transformationEnterprise & Plant

Topology

ERP-LevelEnterprise Resources

MES-LevelManufacturing Execution Systems

Control-LevelMachine Controllers

Device-LevelSensors, Devices

Finance, HR, SCM, SRM, CRM …

Plant Schedules, Products, BOMs,

Routings, Quality, Performance …

HMI, SCADA, Alarms, Events,

Historians …

Automation Control, Safety, …

Information Systems

Responsive and dynamic cross-

industry Value Networks

Cross-plant performance,

coordination & flexibility

Production optimization &

reduced process costs

Intelligent equipment with

reduced failures & downtime

Enhanced

Azure IoT Gateway SDK

On-Premise: Device Connectivity Cloud: Data Ingestion & Processing, Command & Control Cloud: Presentation

Indu

stri

al D

evic

es(O

PC-U

A S

erve

rs)

Hot Path Analytics

Azure Stream Analytics, Azure Storm, …

Azure IoT Hub

OPC Clients, Servers, ERP Portals,

OPC Graph Database and OPC UA .NET Standard Stack

JSON/AMQPUA Binary

Other Devices

OPC UA Client Module

IoT Proxy Module

UA Binary/AMQP

UA Binary

JSON/AMQP

Any

Firewall

Cold Path Analytics & Storage

Azure HD Insight, Azure Storage, SQL, DocDB, …

Presentation & Business Connections

Websites, Mobile Services

Dynamics, BizTalk Services, Notification Hubs

Microsoft

Dynamics

OPC UA Integration into Azure IoT

Azure IoT Connected Factory Architecture

IoT Hub

VM

Linux VM (with multiple assembly lines)

Web App hosting

Solution Dashboard &

OPC UA Client

OPC UA Server

OPC UA Server

OPC UA Server

Gateway SDK with

OPC Proxy &

OPC Publisher

Modules

MES

Simulation

(OPC UA Client)

Telemetry path

Command & Control path

Time Series Insights

Real-Time Analytics

Real-time Analytics

“We are trying to get insights from our devices in real-time, etc.”

Real-time Analytics (aka Stream Analytics) is the phenomenon of processing data as soon as it is generated, to derive very quick analysis/insight for timely action.

I N T E G R A T I O N W I T H A Z U R E E V E N T H U B & I O T H U B

Azure Stream Analytics has built-in, first class integration with Azure Event Hubs and IoT Hub

▪ Data from Azure Event Hubs and Azure IoT Hub can be sources of Streaming Data to Azure Stream Analytics.

• The connections can be established through the Azure Portal without any coding. (see next slide)

▪ Azure Blob Storage is supported as a source of Reference Data.

▪ Azure Stream Analytics supports compression across all data stream input sources (Event Hubs, IoT Hub, and Blob storage).

Azure Blob Storage

Azure IoT Hub

Azure Event Hubs

Reference Data

Streaming Data

Streaming Data

Azure Stream Analytics

S T R E A M I N G - C A N O N I C A L O P E R A T I O N S

B I G D A T A S T R E A M I N G P A T T E R N W I T H A Z U R E

REAL-TIME

APPLICATIONS

REAL-TIME

DASHBOARDSBUSINESS / CUSTOM

APPS

(STRUCTURED)

LOGS, FILES AND MEDIA

(UNSTRUCTURED)

r

SENSORS AND IOT

(UNSTRUCTURED)

EVENT HUBS IoT HUB KAFKA on HDINSIGHT STREAM

ANALYTICS

STORM on

HDINSIGHT AZURE DATABRICKS

(Spark Streaming)

AZURE ML STUDIO

R SERVER AZURE DATABRICKS

(Spark ML)

Advanced Analytics

Advanced Analytics

“We are trying to predict and prevent in advance”

Advanced Analytics is the process of applying machine learning and/or deep learning techniques to data for the purpose of creating predictive/prescriptive insights.

A D V A N C E D A N A L Y T I C S - C A N O N I C A L O P E R A T I O N S

Model Creation & Deployment Process

Collect Data

Prepare Data

Train Model

Evaluate Model

Deploy Model

Insights Actions

Azure IoT Hub

Cloud

GatewayThings

Azure Stream

Analytics

IoT Pattern: Gaining Insight

Consistency

Insights Actions

Azure IoT Hub

Cloud

GatewayThings

IoT Pattern + Edge

Insights

Actions

Azure IoT Edge Deployment

Azure

IoT HubIoT Edge Device

Azure

Machine

Learning

Azure

Stream

Analytics

Azure

Functions

Azure

Cognitive

Services

Azure Container Service

© 2018 Microsoft Corporation. All rights reserved.