9. lecture neural networks - uni-saarland.de · trained by neural networks. a neural network...

9. Lecture

Neural Networks

Application in Automation

Engineering

Soft Control

(AT 3, RMA)

http://www.uni-saarland.de/en/

SC

234 WS 17/18 Georg Frey

Outline of the lecture

1. Introduction to Soft Control: definition and limitations, basics of

"smart" systems

2. Knowledge representation and knowledge processing (Symbolic AI)

Application: expert systems

3. Fuzzy systems: Dealing with fuzzy knowledge

Application: Fuzzy Control

4. Connective systems: Neural Networks

Application: Identification and neural control

1. Basics

2. Learning method

3. Application in Automation Engineering

5. Genetic algorithms: Stochastic Optimization

Application: Optimization

6. Summary & Literature


SC


Contents of 9th Lecture

• Modelling of Systems by NN

Preliminaries

Direct Model

Inverse Model

• Application

Control

“Virtual” Sensors

• Assessment of NN

• Comparison of NN und Fuzzy

• Possible combinations

• Application examples: Load forecasting


SC


Preliminaries

• Neural networks can model any non-linear relations among multiple

input and output variables of a system

• Pure feed-forward networks can only model static relationships

Solution 1: Recurrent Networks

- Training is difficult

Solution 2: External feedback i.e., processing of past values

+ Simple learning algorithm like backpropagation can be used

- The number of past values must be fixed

• Identification with past values: discrete model


SC


Generating the model of a process

• Objective:

Modeling of a process

Networks models the function yk = f(uk-1, yk-1)

For systems of higher order: yk = f(uk-1,uk-2,... ,yk-1,yk-2,...)

• Input:

Current and past values of the process input u

past values of the process output y

• Output:

Current process output yk

• Example


SC


Generating inverse process model

• Objective:

Modeling of inverse process model

Network models the function uk-1 = f(yk, yk-1) or uk-1 = f( yk ,yk-1,yk-2,... uk-2,uk-3,... )

• Inputs:

Current and past values of the process output y

Previous process inputs u

• output:

Current process input uk-1

• Example


SC


Application of the direct model

• Estimation of state variables which are not measurable online to use

in closed-loop controllers (virtual sensor, observers)

Controller Route

NN Model

w u ym

logical

interconnection

yNN

y

-


SC


Application of the inverse model (ideal)

• If the model is ideal it is possible to achieve open-loop control using

inverse model

But:

• Model is not ideal

• There are noises

Route inverse NN Modell w u y


SC


Application of the inverse model (real)

• Use of a controller to remove the noises and to compensate for the

errors in the model

Route inverse NN Model y

Lin. Controller w u

-


SC


Summary of applications of NN in AT (Automation)

• Besides the "classical" tasks such as pattern recognition,

classification, etc. NN can also be used for performing core

functionalities of AT (Automation)

Observer or virtual Sensor

Closed-loop control (in combination with conventional control)

Combinations of the above are also possible

• In addition to the basic structures discussed, there could be many

other structures


SC


Evaluation of NN

• Neural networks can be trained on the basis of data

no modelling of the processes necessary

• Successful applications show the potential of the method

• Knowledge is encoded in the structure of the NN

A verification, interpretation of the calculated values is virtually

not possible (raises acceptance problems!)

• NN training is extensive

• Acquisition of "good" data can be problematic

• To fix the structural parameters, e.g.,

Number of hidden layers

Number of neurons in the hidden layers

Type of network

Type of activation functions

Learning parameters and criteria for stopping training

use of heuristics is preferable in most cases.


SC


Comparison NN vs. Fuzzy

- Often very long computing

times

- Convergence is not ensured

- Long computing

times during

training,

- Many competing

network structures

to choose from

- Extrapolation not

possible, i.e., good

results are achieved

only in the range of

training data

- Knowledge in the

network hardly

interpretable

- Difficult knowledge

acquisition phase

- Optimization phase

often slow

- Unusual way of

thinking

- Application to complex

processes very

cumbersome and

expensive

- Control specialists are

needed to write and

amend the algorithms

- There are scarce

standard tools for

implementing the

algorithms on standard

hardware (e.g., PLC)

+ Like NN but

+ Better interpretation of

knowledge,

+ Knowledge through learning

can gradually be

complemented

+ Adaptive and

adaptable to very

complex dynamic

processes,

+ Possible to retrain

when the process

undergoes changes

+ Simple and

comprehensive form of

algorithms,

+ Easily extensible rule-

base

+ Integration of

knowledge from more

than one source is

possible

+ In-depth process

understanding based on

process analysis

+ Generally the outcome

is very good and optimal

solutions can be

achieved

+ Stability proves are

possible

Neuro-Fuzzy Neuronal Networks Fuzzy Control Classic Method


SC


Approaches for the combination of NN and fuzzy

(A) Cooperative neuro-fuzzy systems: Fuzzy systems which can be trained by neural networks. A neural network connected serially with the fuzzy system can, for example, be used to learn the suitability of a rule in certain situations.

(B) Rule-based training of a simple neural network

(C) Hybrid Neuro-Fuzzy-systems: simple neural networks that uses "fuzzy neurons" (e.g., min-/max-Neurons) and "fuzzy weights". The structure of the fuzzy system can be recognized from the network topology.

(D) Neural networks that can be trained by fuzzy-learning method. The changes of the weights between the neurons is calculated by a fuzzy system at each step.

(E) topological configuration of a neural network, with more or less complex fuzzy systems as neurons.

(F) A mix of classic expert systems and one of the above approaches.

• Important approaches are A, B and C.

• Other approaches are not as widespread the previous ones.


SC


Cooperative neuro-fuzzy systems (2 approaches)

Fine-tuning a fuzzy controller by NN

• A fuzzy controller will be followed by a neural network

• The output of the fuzzy system will be immediately processed by the NN

• Thus based upon a basic knowledge (of the fuzzy system) a non-linear system can be built, which additionally renders adaptability to certain special situations which are not defined by the basic knowledge.

• Thus NN performs the "fine tuning" of output of the fuzzy system. The NN can learn which tuning is necessary for which input.

• The fuzzy system must not deliver defuzzyfied output this task can also be performed by the NN.

Preprocessing the input values of a fuzzy controller by NN

• fuzzy controller is preceded by a NN

• The output of the NN is fed to the fuzzy controller for processing.

• Thus, changes in the input data, which cannot be processed by the fuzzy system can be compensated by NN.


SC


Rule based training of NN

• NN can only be trained by numerical data

• Often a rough knowledge of the process is available in the form of

fuzzy rules

• Solution: mapping of linguistic rules (qualitative) to the training data

(quantitative)

The linguistic terms are mapped to values (according to the membership

functions)

The rules are then defined by the corresponding values

• During training NN interpolates among the values

• Example:

Three variables X1, X2 and Y with values of Small, Medium and Large within the

range of [0, 1] have to mapped to numeric values. It is given that small = 0;

resources = 0.5; Large = 1.

The rule IF X1=small AND X2 = large THEN Y = large

Results in the data set X = (0, 1); Y = 1


SC


Hybrid neuro-fuzzy systems

• Mapping of a fuzzy controller to a neural network

• Example:

1st Layer: input Fuzzy Sets

2nd Layer: evaluate the degree of fulfilment of the rules

3rd Layer: output fuzzy sets

4th Layer: De-fuzzyfication

• Other variants define the fuzzy sets in the weights

• Training with data

• Interpretation of the rules learned as weights (weights between

Layer 1 and 2 or 2 to 3)


SC


Hybrid neuro-fuzzy systems (example)

x1

x2


SC


Example: Load forecast in electrical energy supply networks

• Motivation

• Last curve analysis

• Forecast with Artificial Neural Networks (ANN)

• Wavelet transformation

• Assessment of results

• Summary


SC


Motivation

Structure of an electric power supply network

Power Plant Network

(Low storing capacity)

Consumer

Logic

on/off

deterministic, known not deterministic,

only past behaviour known

PI PO


SC


Motivation

• Last curves forecast plays a major role in the operation of power

networks

Power is cost-effective

Electrical energy is difficult to save

• It should be possible to only produce as much electrical energy as

needed

PI=PO

• Therefore one needs to recorded consumption profiles based

Forecast

Under forecast leads to inadequate provision of spare capacity

Over forecast caused unnecessary spare capacity


SC


Last curve-Analysis

• Network load from 09.06.2003 to 29.06.2003 (individual than three weeks) in the control zone RWE's electricity transportation network

1. From Monday to Sunday, from 0 clock to 24 clock

2. Given are 15-minute averages

3. 4 * 24 = 96 test points per day, 96 * 7 = 672 measuring points per

week

MW


SC


Forecast with artificial neural networks (ANN)

• Forecast runoff

Last curve normalization

Forecast basic idea

KNN-Definition

• Structure, vector input, output vector, activation function

KNN training (with a whole week (this week 1))

• Back propagation-Algorithms

Learning rate

KNN-Application (with Week 2 oder 3)

Results Denormalization

KNN

Modell

( 1, 2,... 8)Lk k k ( 4)L k

Fig 3 : Drei-Schichten-Feed-Forward-Struktur Fig 2 : Einschicht-Neuron


SC



• Last curve-Normalization


SC



KNN

Modell

( 1, 2,... 8)Lk k k ( 4)L k

Three layers feed forward structure

Monolayer neuron Forecast basic idea

1

2

......

8

L k

L k

L k

p 4a Lk

Last course (distribution) of

the last two hours

Last in an hour


SC



• Four-step forecast results

Training of KNN with Week 1

Target vector (SimT): Last curve Week 3

Output vector(Y): Forecast of Week 3


SC


Forecast with artificial neural networks (KNN)

• In many places, the relative error is greater than 10%

The accuracy must be improved

Idea: Installation of Wavelet transformation

Relativer Fehler


SC


Wavelet transformation

• Development of Wavelet transformation Fourier transformation

Transformation from Time- to Frequency Domain

Short-Time-Fourier transformation

Additional Information which Frequency in occurs which time frame

Continuous Wavelet transformation

Transformation of time in frequency and time domain

Discrete Wavelet transformation (DWT)

Realization in Computer

A Trous algorithm of Wavelet transformation

• Shift invariant

• Same in data length in different frequency domains

• suitable for real-time systems

f t Ff

1 2

10

, ,...tt tt

t tt

ft FfFf

,f t Ff


SC


Discrete Wavelet transformation (DWT) (implementation)

• Analysis of a signal

HP

TP

2

High pass filter

Low pass filter

Down sampling

f<fs/16 fs/16<f<fs/8 f<fs/8 fs/8<f<fs/4 f<fs/4 fs/4<f<fs/2 f<fs/2 Frequency response

N/8 N/8 N/4 N/4 N/2 N/2 N Sampling points

a3 d3 a2 d2 a1 d1 x


SC



• Example


SC



• Synthesis of a signal

Upsampling


SC


Discrete wavelet transformation (DWT)

• Requirement of DWT in the analysis of real-time system

Localization time points in different scales

Shift invariance of the system

Move original

curve

Wavelet-

transformation

Wavelet-

Coefficient

Move

Coefficient

Wavelet-

transformation

Wavelet-

Coefficient


SC


Á-Trous algorithm of Wavelet transformation

d1

d2

d3

a3

• Properties of the A-Trous algorithm

Shift invariance

Same data length of all the different scales Wavelet coefficient

g[n] : Tiefpassfilter

h[n] : Hochpassfilter


SC


Wavelet transformation

• Example A-Trous algorithm

Week 1 load curve is split into 4 layers

a4: Approximations signal; d4, d3, d2, d1: detail signals

a4 has the largest amplitude and the lowest frequency

d1 is the smallest and the largest amplitude frequency


SC


Forecast: ANN + A Trous

• Forecast runoff with KNN and A-Trous

For each split signal, a ANN model

The more layers, the higher the accuracy of the load curve synthesis

d1 is the prognosis regarded as noise and neglected.

Recorded

load curves

a4

d4

d3

d2

d1

netA4

netD4

netD3

netD2

netD1

Predicted

Last curve

Wavelet

Re-

transformation

Ã-Trous

Wavelet


SC



• Four-step forecast results

Training with Week1

Target vector(SimT): Last curve Week3

Output vector(Y): Forecast of Week3


SC



• At the most points the relative error less than 2%

• The error is never greater than 6%

In comparison to ANN without A-Trous, the accuracy improved significantly

Relativer Fehler


SC


Summary and learning from the 9th Lecture

Know basic applications of NN in AT

Model shapes in the identification and their target

directly

Inverse

Neural networks with other approaches to (especially fuzzy) compare

Deduce reasons for neuro-fuzzy

Know possible ways of combining NN with fuzzy and can explain the basic idea

Use of neural networks has been shown to predict

Neural networks applied to isolated not bring satisfactory results in the load curve forecasting

In combination with wavelet transform results could be significantly improved


9. lecture neural networks - uni-saarland.de · trained by neural networks. a neural network...

Documents