Machine LearningProven Applications and New Features

Seth DeLand

How to Get Started with Machine Learning?

2

Machine Learning Success Stories

Kinesis Health TechnologiesPredicting a patient’s fall risk with machine learning.

3

Machine Learning+

X4

Machine Learning+

Industry Knowledge Application Knowledge

Your Own Expertise

5

Examples of Successful Machine Learning Applications

Fleet Data Analytics

Energy Forecasting

Manufacturing Analytics

6

New Capabilities§ MATLAB apps§ AutoML§ Signal Processing with

Machine Learning§ C/C++ Code Generation

Examples of Successful Machine Learning Applications

Fleet Data Analytics

Energy Forecasting

Manufacturing Analytics

7

Fleet Data Analytics

8

Design Decisions

Test Plans

0.150.07

Temperature

0 50 100 150 200 250 300 350

Temperature (oF)

0

2

4

6

8

10

12

14

16

Cou

nt

What Level of Data?

Equipment

Signals

Messages

Trip/Session

Time – Value pairs

9

What Type of Question?

10

For each (trip, day, serial #, customer, etc) in the fleet data set, calculate some Key Performance Indicator (KPI*) given parameters XYZ".

Across All (data) in the fleet data set, calculate descriptive statistics of specific variables (min, max, median, count, etc.) to summarize and visualize (histograms).

Question Type “Across All”“For Each”

Scale to Large Collections of Data with Datastore

11

Available DatastoresGeneral datastore

spreadsheetDatastore

tabularTextDatastore

fileDatastore

Database databaseDatastore

Image imageDatastore

denoisingImageDatastore

randomPatchExtractionDatastore

pixelLabelDatastore

augmentedImageDatastore

Audio audioDatastore

Predictive Maintenance

fileEnsembleDatastore

simulationEnsembleDatastore

Simulink SimulationDatastore

Automotive mdfDatastore

Custom subclass matlab.io.Datastore

Transformed transform an existing datastore

Performing “Across All” Calculations with Tall

12

§ Visualizations

§ Data preprocessing

§ Machine Learning

Exploring Fleet Data with Unsupervised Learning

13

Unsupervised Learning for Operational Mode Clustering

14

“Cold Storage” “Hot Storage”

Data

Historic data:• Batch processing• Large data on cluster• Explore long term trends• Build models

Streaming data:• Near real-time• Test and implement model

for new data • Stream processing

Vehicle data, driver profiles

Deploying Fleet Analytics

15

Fleet Analytics Streaming Architecture

16

Fleet Analytics in Practice: Volkswagen Data Lab

Develop technology building block for tailoring car features and services to individual§ Driver and Fleet Safety§ Driver Coaching§ Driver-Specific Insurance

Data sources§ Logged CAN bus data and travel record

Results§ Proof-of-concept model for “telematic fingerprint”§ Basis for the “pay-as-you-drive” concept

Source: “Connected Car – Fahrererkennung mit MATLAB“Julia Fumbarev, Volkswagen Data LabMATLAB EXPO Germany, June 27, 2017, Munich Germany

17

Machine Learning + X

18

Fleet Analytics

Equipment ExpertiseDesign Specs

Operating ModesOperating Conditions

Machine LearningStatistical Analysis

Unsupervised Learning

Energy Forecasting

Electrical Grid ExpertiseSeasonality

Weather EffectsGenerator Characteristics

Machine LearningTime Series Modeling

Regression

Manufacturing Analytics

Manufacturing Expertise

Process EquipmentProcess Variables

Performance Metrics

Machine LearningAnomaly Detection

RegressionClassification

Examples of Successful Machine Learning Applications

Fleet Data Analytics

Energy Forecasting

Manufacturing Analytics

19

The Need for Energy Forecasts

20

Jul 01 Jul 02 Jul 03 Jul 04 Jul 05 Jul 06 Jul 07 Jul 08 Jul 09 Jul 10 Jul 11 Jul 12 Jul 13Time 2019

0

1

2

3

4

5

6

7

8

9

10

Win

d Sp

eed

(m/s

)

Wind

Aug 04 Aug 05 Aug 06 Aug 07 Aug 08 Aug 09 Aug 10 Aug 11Time 2019

2000

2500

3000

3500

4000

4500

5000

5500

6000

6500

Load

(MW

)

Demand

Price

Jun 01 Jun 02 Jun 03 Jun 04 Jun 05 Jun 06 Jun 07 Jun 08 Jun 09 Jun 10 Jun 11 Jun 12Time of day 2018

0

50

100

150

200

250

Ener

gy p

rodu

ctio

n (k

W)

Solar

How Energy Forecasting Works

21

Electricity Demand

Weather

Electricity Prices

Combine

Historical Data

Preprocess FeaturesMachine Learning

Jul 01 Jul 02 Jul 03 Jul 04 Jul 05 Jul 06 Jul 07 Jul 08 Jul 09 Jul 10 Jul 11 Jul 12 Jul 13Time 2019

0

1

2

3

4

5

6

7

8

9

10

Win

d Sp

eed

(m/s

)

Aug 04 Aug 05 Aug 06 Aug 07 Aug 08 Aug 09 Aug 10 Aug 11Time 2019

2000

2500

3000

3500

4000

4500

5000

5500

6000

6500

Load

(MW

)

loadtemp

wind

day month

24hr1week

Building Forecast Models with Regression Techniques

22

Using Energy Forecasting Models

23

New Data

Electricity Demand

Weather

Electricity Prices

Jul 01 Jul 02 Jul 03 Jul 04 Jul 05 Jul 06 Jul 07 Jul 08 Jul 09 Jul 10 Jul 11 Jul 12 Jul 13Time 2019

0

1

2

3

4

5

6

7

8

9

10

Win

d Sp

eed

(m/s

)

Aug 04 Aug 05 Aug 06 Aug 07 Aug 08 Aug 09 Aug 10 Aug 11Time 2019

2000

2500

3000

3500

4000

4500

5000

5500

6000

6500

Load

(MW

)

Combine Features Trained Machine Learning

Model

Forecastloadtemp

wind

day month

24hr1week

Deploying Energy Forecasts

Dashboards for operators and traders

24

API for App Developers

Combining Forecasting with Optimization

“When should I operate my generators to maximize the return on my investment?”

25

Optimization Problem:

Minimize: Cost of generating electricity

Constraints: 1) Meet forecasted demand2) Operational constraints3) Etc.

ChallengeMaximize margins in energy trading by predicting available supply and peak demand

SolutionUse MATLAB to build and optimize models that incorporate historical data, weather forecasts, and regulatory rules

Results§ Response time reduced by months§ Productivity doubled§ Program maintenance simplified

“Because we need to rapidly respond to shifting production constraints and changing demands, we cannot depend on closed or proprietary solutions. With MathWorks tools we get more accurate results — and we have the flexibility to develop, update, and optimize our models in response to changing needs.”- Angel Caballero, Gas Natural FenosaLink to user story

Portomouros hydroelectric dam.

Energy Forecasting in Practice: Naturgy Energy Group S.A.

26

Machine Learning + X

27

Fleet Analytics

Equipment ExpertiseDesign Specs

Operating ModesOperating Conditions

Machine LearningStatistical Analysis

Unsupervised Learning

Energy Forecasting

Electrical Grid ExpertiseSeasonality

Weather EffectsGenerator Characteristics

Machine LearningTime Series Modeling

Regression

Manufacturing Analytics

Manufacturing Expertise

Process EquipmentProcess Variables

Performance Metrics

Machine LearningAnomaly Detection

RegressionClassification

Machine Learning apps

§ Try out many models§ Compare Results§ Get to a reasonable model

without worrying about the details

28

Perform Hyperparameter

Optimization in apps

AutoML

§ Build many machine learning models§ Find a good model without becoming

an expert

29

Preprocess Data

Deploy & Integrate

Train Model

Extract Features

Import Data

Wavelet Scattering

Decision Tree?SVM?KNN?Ensemble?…?

fitcauto

Wavelet Scattering

Hyper-parameter

Optimization

Feature Selection

Model Selection

Wavelet Scattering

AutoML “in action”

30

Examples of Successful Machine Learning Applications

Fleet Data Analytics

Energy Forecasting

Manufacturing Analytics

31

What is Manufacturing Analytics?

Definition: Apply modeling (AI) to process and sensor data to maximize operational performance

Key Use Cases:1. Automate the monitoring of manufacturing process2. Ensure product quality3. Optimize yield of complex production processes

32

Challenges in Applying AI to Manufacturing

Lots of Data – much in “Data Historians” (SCADA, LIMS, OSISoft PI)

Reliable measurements or modeling– Sensor failures– Hidden variables

Use of many different tools– Limited Predictive modeling– Handle streaming data– Customization

33

Uncover Hidden Variables with Process Modeling

34

Catalyst Aging

pretty big à

Case Study: Anomaly Detection

35

Case Study: Anomaly Detection

2. One-class SVM1. Cluster with DBSCAN

36

Deployment

Integration with Data Historians§ OPC Toolbox (Database tbx via ODBC

or JDBC) connects with PI Server§ Access MATLAB analytics within the

PI Asset Framework

Customize Analytics Delivery§ Accessing insights via

GUI critical for plant staff and process engineers

§ Build a custom dashboard with App Designer

37

PI Data Archive

OPC, ODBC, JDBC

MATLAB Production Server

Calls MATLAB function

PI Asset Framework

Write result to PI point

PI System Explorer

Machine Learning + X

38

Fleet Analytics

Equipment ExpertiseDesign Specs

Operating ModesOperating Conditions

Machine LearningStatistical Analysis

Unsupervised Learning

Energy Forecasting

Electrical Grid ExpertiseSeasonality

Weather EffectsGenerator Characteristics

Machine LearningTime Series Modeling

Regression

Manufacturing Analytics

Manufacturing Expertise

Process EquipmentVariables & Set Points

Parameter Impact

Machine LearningAnomaly Detection

RegressionMultivariate Statistics

Machine Learning + Signal Processing

39

Data Preprocessing Feature Engineering

Frequency domain

Time domain

Bandwidth measurements Spectral statisticsDetrending Smoothing

Resampling Filtering

Kinesis Health Technologies

Predicting a patient’s fall risk with machine learning.

40

From Desktop to Production

Reasons for Updates:§ Found a better model§ New data became available§ Business needs change§ …

41

C/C++

Automatic C/C++ Code Generation

1. Prediction for most Classification and Regression models

2. Update deployed models without regenerating code– SVM, Decision Trees, Linear Models

1. Fixed-Point support– SVM, Decision Trees, Ensemble of Trees– Shallow Neural Network (through Simulink)

1. Integrate with Simulink models asMATLAB Function Block

42

Integrate MATLAB with Other Languages

Examples of Successful Machine Learning Applications

Fleet Data Analytics

Energy Forecasting

Manufacturing Analytics

43

New Capabilities§ MATLAB apps§ AutoML§ Signal Processing with Machine Learning§ C/C++ Code Generation

Machine Learning+

XFleet Data Analytics

Energy ForecastingManufacturing Analytics

Signal Processing

Industry Knowledge

Application KnowledgeMedical Devices Mining

AppsC/C++ Code GenerationAutoML

44

Learn More

- Exploratory Data Analysis- Data Processing and Feature Engineering- Predictive Modeling and Machine Learning- Data Science Project

Training Courses

MATLAB Fundamentals (3 days)

MATLAB for Data Processing and Visualization (1 day)

Processing Big Data with MATLAB (1 day)

Statistical Methods in MATLAB (2 days)

Machine Learning with MATLAB (2 days)

Signal Preprocessing and Feature Extraction with MATLAB (1 day)

Deep Learning with MATLAB (2 days)

Accelerating and Parallelizing MATLAB Code (2 days)

45