improving long- term learning of model reference adaptive...

Improving Long-Term Learning of Model Reference Adaptive

Controllers for Flight Applications: A Sparse Neural Network Approach

AIAA Guidance, Navigation, and Control Conference January 2017

Scott A. Nivison Pramod P. Khargonekar

Department of Electrical and Computer Engineering University of Florida

Distribution A: Approved for public release; distribution is unlimited.

Outline

Motivation Prior Research MRAC Formulation Unstructured Neural Network (SHL) Structured Neural Network (RBF) Sparse Neural Network Approach (SNN) Simulation Results Future Research Goals

Distribution A: Approved for public release; distribution is unlimited

Motivation

Highly Dynamic Flight Vehicles Trade-offs: Number of Nodes and Learning Rates

Sparse Learning Sparse Auto-encoders Sparse Activation Function: Linear Rectifier Sparse Optimization Techniques: Max-out and

Channel-out

Long-Term Learning Performance: Uncertainty Estimates


Prior Research

Develop a MRAC architecture that improves long-term learning and tracking performance of flight vehicles with consistent uncertainties over regions of the flight envelope while utilizing small to moderate learning rates and significant processing constraints.

Research Goals

Enhancements to the MRAC architecture 𝐿1 Adaptive Control Concurrent Learning


MRAC Formulation

System Dynamics:

�̇� = 𝐴𝑥 + 𝐵Λ 𝑢 + 𝑓 𝑥 + 𝐵𝑟𝑟𝑟𝑦𝑐𝑐𝑐 𝑦 = 𝐶𝑥

𝐴 ∈ ℝ𝑛×𝑛,𝐵 ∈ ℝ𝑛×𝑐, 𝐵𝑟𝑟𝑟 ∈ ℝ𝑛×𝑐, 𝐶 ∈ ℝ𝑝×𝑛 are constant known matrices

Λ ∈ ℝ𝑐×𝑐 constant unknown diagonal matrix 𝑓(𝑥) ∈ ℝ𝑐 unknown continuous differentiable function

𝑦𝑐𝑐𝑐 ∈ ℝ𝑐 external bounded time-varying command

Reference Model: �̇�𝑟𝑟𝑟 = 𝐴𝑟𝑟𝑟𝑥𝑟𝑟𝑟 + 𝐵𝑟𝑟𝑟𝑦𝑐𝑐𝑐

𝑦𝑟𝑟𝑟 = 𝐶𝑟𝑟𝑟𝑥𝑟𝑟𝑟

𝐴𝑟𝑟𝑟 ∈ ℝ𝑛×𝑛, 𝐶𝑟𝑟𝑟 ∈ ℝ𝑝×𝑛 are constant known matrices 𝐴, 𝐵Λ is controllable


Aref= 𝐴 − 𝐵𝐾𝐿𝐿𝐿

MRAC – Adaptive Augmentation

𝑢 = 𝑢𝐵𝐿 + 𝑢𝐴𝐴 Overall Control:

𝑢𝐵𝐿 = −𝐾𝐿𝐿𝐿𝑥, 𝑥 = (𝑒𝐼 , 𝑥𝑝) ∈ ℝ𝑛 𝑒�̇� = 𝑦 − 𝑦𝑐𝑐𝑐


MRAC Formulation

State Tracking Error:

𝑒 = 𝑥 − 𝑥𝑟𝑟𝑟

Adaptive Controller:

�̇� = 𝐴𝑟𝑟𝑟𝑒 + 𝐵Λ(𝑢𝐴𝐴 + 𝑓 𝑥 + 𝐼 − Λ−1 𝑢𝐵𝐿)

𝑢𝐴𝐴 = −𝐾�𝐵𝐿(−𝐾𝐿𝐿𝐿𝑥) − Θ�TΦ(𝑥)

Tracking Objective: limt→∞

𝑥 𝑡 − 𝑥𝑟𝑟𝑟 𝑡 ≤ 𝜖

Θ� ∈ ℝ 𝑁+1 ×𝑐, Φ 𝑥 ∈ ℝ 𝑁+1 are matrices of NN weights


Unstructured Neural Network (SHL)


𝑓 𝑥 = 𝑊𝑇𝜎 𝑉𝑇𝜇 + 𝜖

𝑊 = 𝑊𝑇 𝑏𝑊 𝑇 ∈ ℝ(𝑁+1)×𝑐

NN Approximation Theorem:

𝑉 = 𝑉𝑇 𝑏𝑉 𝑇 ∈ ℝ(𝑛+1)×𝑁


𝑢𝐴𝐴 = −𝑊� 𝑇𝜎 𝑉�𝑇𝜇 , 𝜇 ∈ ℝ𝑛+1

Adaptive Update Laws:

𝑊�̇ = 𝑃𝑃𝑃𝑃(2Γ𝑊((𝜎(𝑉�𝑇𝜇)-�̇�(𝑉�𝑇𝜇) 𝑉�𝑇𝜇)𝑒𝑇𝑃𝐵) 𝑉�̇ = 𝑃𝑃𝑃𝑃(2Γ𝑉𝜇𝑒𝑇𝑃𝐵𝑊� 𝑇�̇�(𝑉�𝑇𝜇)) Distribution A: Approved for public release; distribution is unlimited.

Structured Neural Network (RBF)



𝑢𝐴𝐴 = −𝑊� 𝑇𝜙 𝑥

Adaptive Update Law:

𝑊�̇ = 𝑃𝑃𝑃𝑃 Γ𝑊𝜙 𝑥 𝑒𝑇𝑃𝐵

𝜙 𝑥 = 𝜙1 𝑥 , … ,𝜙𝑁 𝑥 , 1 𝑇 ∈ ℝ𝑁+1 is a vector of 𝑁 RBFs 𝑊� ∈ ℝ(𝑁+1)×𝑐 are the outer layer weights

𝜙𝑖 𝑥 = 𝑒𝑥−𝑥𝑐

2

2𝜎𝐷 , ∀𝑖 = 1, … ,𝑁 𝑥𝑐 is the fixed center 𝜎𝐴 is the RBF width


Sparse Neural Network (SNN)




Segment flight

envelope into regions and distribute a specified number of nodes to each region.

Activate only a small percentage of the total number of nodes for control at each point in the operating envelope



Adaptive Controller: 𝑃 = 𝑝1, … ,𝑝𝑇 𝑆 = 𝑠1, … , 𝑠𝑇 𝑇 ∈ ℕ 𝑇 is the total number of segments 𝐼 = {1, … ,𝑇}

𝑠𝑖 = {𝑥𝑜𝑝 ∈ 𝑋: 𝐷 𝑥𝑜𝑝,𝑝𝑖 ≤ 𝐷 𝑥𝑜𝑝,𝑝𝑗 ∀𝑖 ≠ 𝑃}

𝐷:𝑋 × 𝑋 → ℝ 𝑥𝑜𝑝 ∈ ℝ𝑁

SNN Definitions:

Metric Space 𝑋,𝐷 :




Adaptive Controller: 𝑁 ∈ ℕ is the total number of nodes 𝑄 ∈ ℕ where 𝑄 = 𝑁

𝑇

𝑄 is the number of nodes per segment

𝐸𝑖∈𝐼 = {𝑒𝐿 𝑖−1 +1, … , 𝑒𝑖𝐿} where 𝑌 =∪𝑖∈𝐼 𝐸𝑖

𝑌 = 𝑒1, … , 𝑒𝑁 𝐵 = {1, … ,𝑁}

SNN Definitions:

Adaptive Controller: Nodes per Segment:




Adaptive Controller: 𝐸𝐴𝑖 = 𝐸𝑖 ∀𝑖 ∈ 𝐼

Pure Sparse Approach (R=Q):

Adaptive Controller: Blended Approach (R>Q):

𝑅 ∈ ℕ is the number of active nodes

𝐸𝐴𝑖 ⊆ 𝐸𝑖 ∀𝑖 ∈ 𝐼


Each segment activates only nodes that were allocated to that segment

Each segment activates R nearby nodes regardless of node segment assignment





Adaptive Update Laws:

𝑢𝐴𝐴 = −𝑊𝑖� 𝑇𝜎 𝑉� 𝑖𝑇𝜇

𝑊𝑖�̇ = 𝑃𝑃𝑃𝑃(2Γ𝑊((𝜎(𝑉𝑖�𝑇𝜇)-�̇�(𝑉𝑖�

𝑇𝜇) 𝑉� 𝑖𝑇𝜇)𝑒𝑇𝑃𝐵)

𝑉𝑖�̇ = 𝑃𝑃𝑃𝑃(2Γ𝑉𝜇𝑒𝑇𝑃𝐵𝑊𝑖� 𝑇�̇�(𝑉𝑖�

𝑇𝜇))


Simulation

Adaptive Controller: Longitudinal Short-Period Dynamics for High-Speed Flight Vehicle:

Adaptive Controller: Flight Condition:


Results - LQR


Results - SHL


Results - RBF


Results – RBF vs SHL


Results - SNN

Adaptive Controller: Longitudinal Short-Period Dynamics for High-Speed Flight Vehicle:



Since the disturbance, 𝑓(𝑥) , was designed based on a single input variable, 𝛼 , only the 1-D SNN architecture with T=91 segments was employed for simulation results.

Results – SNN


Results and Conclusions


Conclusions

Traditional Neural Network schemes typically update adaptive weights based solely on the current state vector which leads to poor long-term learning Sparse Neural Network (SNN) adaptive controllers only update a small portion of neurons at each point in the flight envelope Better memory for uncertainty estimates and weights

from previously visited segments Superior tracking performance and uncertainty

estimates for tasks that have consistent uncertainties and disturbances over regions of the flight envelope


Future Work

Develop standard analysis tools to explore trade-offs between variations of neural network adaptive controllers Explore effectiveness of high dimensional sparse

neural network (SNN) adaptive controllers against numerous uncertainties

Investigate structured sparse neural networks (SNN) for adaptive control of flight vehicles


Questions?


improving long- term learning of model reference adaptive...

Documents