Neural Network Based Online Optimal Control of Unknown MIMO Nonaffine Systems with Application to HCCI Engines

OBJECTIVES

Develop an optimal control of MIMO nonlinear nonaffine discrete-time systems is considered when the dynamics are unknown since the optimal control of multi-input and multi-output (MIMO) nonaffine systems is a challenging problem due to the presence of inputs inside the system dynamics and potentially complex interactions among states and inputs.

Develop an innovative online identifier to provide the gain matrix.Apply the forward-in-time optimal control approach to minimize the

cycle-by-cycle dispersion of a discrete-time representation of the experimentally validated HCCI engine model.

Student: Hassan Zargarzadeh, PhD Student, ECE Department

BACKGROUNDSeveral optimal control approaches for linear and nonlinear systems are

presented with offline techniques since 1985.In the field of optimal control of nonlinear affine systems, a NN online

approach (Dierks and Jagannathan 2009) is proposed.Lewis et al 2010 has done the work using a LMI based approach for

optimal control of unknown linear discrete-time and continuous-time systems. Proposed approach uses the work of Yang et al. 2007 to convert the

nonlinear nonaffine into an affine-like equivalent system. Subsequently, an approach similar to Dierks and Jagannathan 2009 will be employed.

NN-BASED CONTROL APPROACH Under some certain conditions, it can be shown that the control of the nonaffine nonlinear system can

expressed as state feedback control of an affine-like equivalent system as

The affine-like dynamics are completely unknown and an initial admissible controller is necessary for the NN to keep the system stable while the NN learns.

For the optimal controller design, the HJB approach is used that for minimizing a cost function as

.The optimal controller can be obtained as

In the above, , , and a have to be estimated. Therefore, three separate neural network based estimators and system identifier are utilized

Faculty Advisor: Prof. Jagannathan Sarangapani, ECE Department

RESULTSApplication to the HCCI Engine

• The NN-Based optimal controller is applied the MIMO Engine Dynamics.

DISCUSSION For proving the overall closed-loop stability, Lyapunov analysis is utilized and

demonstrated (not shown here).

Fig. 1 shows the proposed control approach while Fig. 2 presents the performance of the initial stabilizing and suboptimal controllers for an operating point. On the other hand, Fig. 3 shows the performance in terms of the one of the outputs.

Fig. 4 shows the convergence of the neural network identifier in estimating the control input gain matrix

Fig. 5 and 6 illustrate the initial admissible and suboptimal control inputs and performance of the controllers respectively in reducing cyclic dispersion.

Finally, Figs 7 and 8 depict the performance of the controllers with time and by using return maps. Fig. 7 clearly shows that the performance of the proposed suboptimal controller is significantly better in controlling the HCCI engine when compared to an open loop and initial stabilizing controllers.

CONCLUDING REMARKSProposed online near (sub) optimal control of unknown non-

affine systems provides an accurate and acceptable control of an nonaffine nonlinear system.

The stability of the closed loop system is shown under the assumption that an admissible controller is given and updated until it converges to an admissible optimal controller.

The optimal controller is proven to be stable both analytically and on an experimentally validated HCCI engine model uses three neural networks: 1) the cost approximation neural network; 2) the optimal control input neural network; and 3) the neural network identifier.

As an application, the approach is applied to the MIMO nonaffine model of an experimentally validated HCCI engine. The simulation results show a significant reduction in the control effort and cyclic dispersion when the suboptimal controller applied.

FUTURE WORK

The system identification process should be improved for better accuracy.

The index function estimation may not converge by just using a one layer-neural network so a multilayer neural network must be employed.

The admissibility of the updated control law should be investigated as a possible future step.

The authors acknowledge the technical support and assistance of Dr. Drallmeier, Joshua Bettis (Mechanical Eng. Department), and Dr. Dierks (MST alumni 2009).

Acknowledgements

NN-Optimal Controller

NN-Index Function Estimator

NN-System Identifier

Fig. 1. The proposed control system approach.

0.54 0.55 0.56 0.57 0.58 0.59 0.6 0.61 0.62 0.63361

362

363

364

365

366

367

368

P3 (KN/cm2)

23 (

CAD

)

Convergence with the suboptimal controller

Convergence with the nonoptimal controller

Fig. 2. Convergence of the closed loop system at the operating point with the initial admissable and suboptimal controllers.

0 100 200 300 400 500 600 700 800 900 10000

200

400

600

Iterations (k)

r

k

Initial admissible controller

Sub-optimal controller

Fig. 3. Comparison between initial admissable and suboptimal controllers where the operating point is .

0 50 100 150 200 250 300 350 4000

0.5

1

1.5

2

Iteration (k)

Gk

Gk11

X 103

Gk12

X 105

Gk21

Gk22

X 103

Fig. 4 Convergence of during the process.

0 50 100 150 200 250 300 350 400

0

50

100

T

in

(a)

Sub-optimal controller

Initial admissible controller

0 50 100 150 200 250 300 350 400-1

-0.5

0

0.5

1

1.5(b)

Iterations(k)

k

Sub-optimal controller

Initial admissible controller

Fig. 5. A comparison between the system input of the initial admissible controller and the optimal controller.

0.5 0.51 0.52 0.53 0.54 0.55 0.56 0.57 0.58 0.59 0.6362

363

364

365

366

367

368

k

Tin

Sup-optimal controller

Initial admissible controller

Fig. 6 Comparison between the initial admissible and the suboptimal controllers.

0 200 400 600 800 1000 12000

0.5

1

Iterations (k)

P3 (

KN

/cm

2 )

0 200 400 600 800 1000 1200

360

380

400

420

23 (

CA

D)

Fig. 7. Comparison between open loop, admissible, and the sub-optimal controller when the setpoint is : the controller switches from open-loop to admissible at k=400; then, to the sub-optimal controller at k=800.

0 0.2 0.4 0.6 0.8 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

P3(k)

P3(k

+1)

360 380 400 420 440350

360

370

380

390

400

410

420

430

23

(k)

23(k

+)

Open loop

Closed admissible loopSub-optimal controller

Fig. 8. Return map of the peak pressure and the crank angle regarding to the comparison made in Figure 7; the sub-optimal controller converges to the setpoint with a transient behavior.

ˆ ( )G K

3 230.55, 370P

23 3( , ) (0.55,365)P

23 3( , ) (0.55,365)P

The Intake Temp ( )inT k

Lean Equivalence Ratio k

The Crank Angle

Maximum Pressure 3 ( )P k

23 ( )k

1 ( ) ( )k k k kX F X G X u

1( ) ( ) ( )Tk k k k kJ X Q X u R u J X

1

*

* 1

1

1( )

2kT

k kk

Ju R G X

X

*ku 1

*

kJ

( )kG X

1 ( )Tk k k kX X W U

1ˆ ˆ( )Tk k k kX X W U

** 1 1

1

1 ˆˆ ˆˆ ˆ ( )2

T

T T kk k k k uk k k uk

k

u u R G XX

ˆ ˆ ( )Tk k kJ X

1 1 ( ) Tk k k k kW W E X U

11 2

1

ˆ ˆ1

Tk k

k k u

k

u

ku k

System Inputs System Outputs

1

1 ( )

k

Tk k k k

W

W E X U

ˆˆ Tk k ku

11 2

1

ˆ ˆ1

Tk k

k k u

k

u

ˆ ˆ ( )Tk k kJ X

Desired Value

ˆk

ˆ ( )kG X

1 ( , )k k kx f x u

( , )k k ky g x u

NN-BASED CONTROLLER DESIGN The identification block uses a NN to estimate the gain matrix necessary in control signal. The Innovative

MIMO system identifier and its weight update law is given as follows:

The index function should also be estimated using another NN which done by (Travis Dierk 2009) with a NN estimator as . .

Finally, the forward in time optimal control input to the nonaffine system is defined as the following NN

and the update law is proposed as

The block diagram representation of the proposed controller is shown below.

Block diagram representation of the proposed controller