(co)-algebraic and analytical aspects of weighted automata ... · minimisation vs approximation •...
TRANSCRIPT
![Page 1: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/1.jpg)
(Co)-Algebraic and Analytical Aspects of Weighted Automata Minimisation and Equivalence
(Part 2)
Borja Balle
[CMCS Tutorial, Apr 2 2016]
![Page 2: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/2.jpg)
Minimisation vs Approximation
16 novática Special English Edition - 2013/2014 Annual Selection of Articles
monograph Process Mining
monograph
Figure 6. Process Model Discovered for a Group of 627 Gynecological Oncology Patients.
Figure 5. Conformance Analysis Showing Deviations between Eventlog and Process.
16 novática Special English Edition - 2013/2014 Annual Selection of Articles
monograph Process Mining
monograph
Figure 6. Process Model Discovered for a Group of 627 Gynecological Oncology Patients.
Figure 5. Conformance Analysis Showing Deviations between Eventlog and Process.
minimisation
![Page 3: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/3.jpg)
Minimisation vs Approximation
16 novática Special English Edition - 2013/2014 Annual Selection of Articles
monograph Process Mining
monograph
Figure 6. Process Model Discovered for a Group of 627 Gynecological Oncology Patients.
Figure 5. Conformance Analysis Showing Deviations between Eventlog and Process.
16 novática Special English Edition - 2013/2014 Annual Selection of Articles
monograph Process Mining
monograph
Figure 6. Process Model Discovered for a Group of 627 Gynecological Oncology Patients.
Figure 5. Conformance Analysis Showing Deviations between Eventlog and Process.
minimisation
q1 q2
q3 q4approximation
![Page 4: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/4.jpg)
Minimisation vs Approximation
• Minimisation is naturally related to (co-)algebraic properties
• Approximation questions need analytical tools (eg. metrics)
16 novática Special English Edition - 2013/2014 Annual Selection of Articles
monograph Process Mining
monograph
Figure 6. Process Model Discovered for a Group of 627 Gynecological Oncology Patients.
Figure 5. Conformance Analysis Showing Deviations between Eventlog and Process.
16 novática Special English Edition - 2013/2014 Annual Selection of Articles
monograph Process Mining
monograph
Figure 6. Process Model Discovered for a Group of 627 Gynecological Oncology Patients.
Figure 5. Conformance Analysis Showing Deviations between Eventlog and Process.
minimisation
q1 q2
q3 q4approximation
![Page 5: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/5.jpg)
Motivations for Approximation
1. Efficient (approximate) computation with WA
• Compute faster and pay a controlled price in terms of accuracy
2. Machine learning for WA
• Understand learning bias, design better algorithms
3. Interesting mathematical questions
![Page 6: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/6.jpg)
WA and Rational Functions
A q1
11
q2
´1´2
a, 1b, 0
a,´1b,´2
a, 3b, 5
a,´2b, 0 A = ��, �, {Ta}a���
�, � � Rn, Ta � Rn�n
� =
1-1
�Tb =
�0 �20 5
�Ta =
�1 �1
�2 3
�� =
�1 �2
�
![Page 7: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/7.jpg)
WA and Rational Functions
A q1
11
q2
´1´2
a, 1b, 0
a,´1b,´2
a, 3b, 5
a,´2b, 0 A = ��, �, {Ta}a���
�, � � Rn, Ta � Rn�n
fA : ⌃? ! Rsum of weights of all paths labelled by the
input string
fA(x) = �Tx1 · · · Txt�
= �Tx�
![Page 8: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/8.jpg)
(Pseudo-)Metrics for ApproximationHow do we measure the quality of WA approximations?
![Page 9: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/9.jpg)
(Pseudo-)Metrics for Approximation
�f�p =
��
x���
|f(x)|p�1/p
distp(A,B) = ∥fA − fB∥p
How do we measure the quality of WA approximations?
![Page 10: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/10.jpg)
(Pseudo-)Metrics for Approximation
�f�p =
��
x���
|f(x)|p�1/p
distp(A,B) = ∥fA − fB∥p
How do we measure the quality of WA approximations?
pseudo-metric
![Page 11: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/11.jpg)
(Pseudo-)Metrics for Approximation
�f�p =
��
x���
|f(x)|p�1/p
distp(A,B) = ∥fA − fB∥p
How do we measure the quality of WA approximations?
ℓp = {f : Σ⋆ → R | ∥f∥p < ∞}
Rat = {f : Σ⋆ → R | rational}
ℓpRat = Rat ∩ ℓp = {f : Σ⋆ → R | rational, ∥f∥p < ∞}
Important linearsubspaces of R��
![Page 12: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/12.jpg)
(Pseudo-)Metrics for Approximation
�f�p =
��
x���
|f(x)|p�1/p
distp(A,B) = ∥fA − fB∥p
How do we measure the quality of WA approximations?
ℓp = {f : Σ⋆ → R | ∥f∥p < ∞}
Rat = {f : Σ⋆ → R | rational}
ℓpRat = Rat ∩ ℓp = {f : Σ⋆ → R | rational, ∥f∥p < ∞}
Important linearsubspaces of R��
Banach/Hilbert
not complete!
![Page 13: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/13.jpg)
An Approximation Result
There is an algorithm that given a minimal WA A with nstates and k < n, runs in time O(|�|n6) and returns aWA A with k states such that
dist2(A, A) ��
�2k+1 + · · · + �2
n
[Balle, Panangaden, Precup 2015, Kulesza, Jiang, Singh 2015]
![Page 14: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/14.jpg)
An Approximation Result
There is an algorithm that given a minimal WA A with nstates and k < n, runs in time O(|�|n6) and returns aWA A with k states such that
dist2(A, A) ��
�2k+1 + · · · + �2
n
singular values of theHankel matrix of fA
[Balle, Panangaden, Precup 2015, Kulesza, Jiang, Singh 2015]
![Page 15: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/15.jpg)
The Plan
1. Hankel Matrices and Singular Value Automata (SVA)
2. Algorithms for Computing SVA
3. WA Approximation via SVA Truncation
![Page 16: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/16.jpg)
The Hankel Matrix
Hf 2 R�?⇥�?
f : �? ! R
Hf(p, s) = f(ps)
![Page 17: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/17.jpg)
»
——————————–
" a b ¨¨¨ s ¨¨¨
" fp"q fpaq fpbq ...
a fpaq fpaaq fpabq ...
b fpbq fpbaq fpbbq ......p ¨ ¨ ¨ ¨ ¨ ¨ ¨ ¨ ¨ fppsq...
fi
����������fl
The Hankel Matrix
Hf 2 R�?⇥�?
f : �? ! R
Hf(p, s) = f(ps)
![Page 18: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/18.jpg)
»
——————————–
" a b ¨¨¨ s ¨¨¨
" fp"q fpaq fpbq ...
a fpaq fpaaq fpabq ...
b fpbq fpbaq fpbbq ......p ¨ ¨ ¨ ¨ ¨ ¨ ¨ ¨ ¨ fppsq...
fi
����������fl
The Hankel Matrix
[Kronecker 1881, Carlyle & Paz 1971, Fliess 1974]
Theorem: rank(Hf) = n iff f computed by minimal WA with n states
Hf 2 R�?⇥�?
f : �? ! R
Hf(p, s) = f(ps)
![Page 19: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/19.jpg)
Hf(p, s) = f(ps) = αTp1 · · · TptTs1 · · · Tst′β = αp ·βs
Proof (upper bound rank(HfA) � |A|)
![Page 20: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/20.jpg)
Structure of Low-rank Hankel Matrices
HfA R� � P R� n S Rn �
s
...
...
...p
...
p
s
fA p1 pT s1 sT �0 Ap1 ApT
↵A p
As1 AsT �
�A s
�A p P p, ⇥A s S , s
Hf(p, s) = f(ps) = αTp1 · · · TptTs1 · · · Tst′β = αp ·βs
Proof (upper bound rank(HfA) � |A|)
![Page 21: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/21.jpg)
Structure of Low-rank Hankel Matrices
HfA R� � P R� n S Rn �
s
...
...
...p
...
p
s
fA p1 pT s1 sT �0 Ap1 ApT
↵A p
As1 AsT �
�A s
�A p P p, ⇥A s S , s
Hf = P · S
Hf(p, s) = f(ps) = αTp1 · · · TptTs1 · · · Tst′β = αp ·βs
Proof (upper bound rank(HfA) � |A|)
![Page 22: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/22.jpg)
Proof (equality rank(Hf) = minfA=f |A|)1. Suppose Hf = P · S is rank factorisation
2. For a � � define Haf (p, s) = f(pas) and note
Haf = RaHf where (Raf)(x) = f(xa)
3. Observe that because S has full row rank colspan(RaP) =colspan(Ha
f ) � colspan(Hf) = colspan(P)
4. Hence there exists Ta such that RaP = PTa
5. Let A = �� = P(�, �), � = S(�, �), {Ta}�
6. Note fA = f since P(�, �)TxS(�, �) = RxP(�, �)S(�, �) =P(x, �)S(�, �) = Hf(x, �)
![Page 23: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/23.jpg)
A Useful Correspondence
AQ = ��Q, Q�1�, {Q�1TaQ}a���
A = ��, �, {Ta}a��� Q � Rn�n invertible
![Page 24: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/24.jpg)
A Useful Correspondence
(�Q)(Q�1Tx1Q) · · · (Q�1TxtQ)(Q�1�) = �Tx1 · · · Txt�
AQ = ��Q, Q�1�, {Q�1TaQ}a���
A = ��, �, {Ta}a��� Q � Rn�n invertible
![Page 25: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/25.jpg)
A Useful Correspondence
(�Q)(Q�1Tx1Q) · · · (Q�1TxtQ)(Q�1�) = �Tx1 · · · Txt�
AQ = ��Q, Q�1�, {Q�1TaQ}a���
A = ��, �, {Ta}a��� Q � Rn�n invertible
Theorem:
bijection
For any rational f : �� � R
Minimal WA Acomputing f
Rank factorizationsof the form Hf = P · S
[Balle, Carreras, Luque, Quattoni 2014]
![Page 26: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/26.jpg)
SVD of Hankel Matrices
Hf = UDV⊤ =
⎡
⎢⎢⎣
......
u1 · · · un
......
⎤
⎥⎥⎦
⎡
⎢⎣σ1
. . .σn
⎤
⎥⎦
⎡
⎢⎣· · · v⊤1 · · ·
...· · · v⊤n · · ·
⎤
⎥⎦
![Page 27: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/27.jpg)
SVD of Hankel Matrices
Hf = UDV⊤ =
⎡
⎢⎢⎣
......
u1 · · · un
......
⎤
⎥⎥⎦
⎡
⎢⎣σ1
. . .σn
⎤
⎥⎦
⎡
⎢⎣· · · v⊤1 · · ·
...· · · v⊤n · · ·
⎤
⎥⎦
Theorem:
Hf with f rational admitsan SVD iff �f�2 < �
[Balle, Panangaden, Precup 2015, 201?]
![Page 28: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/28.jpg)
SVD of Hankel Matrices
Hf = UDV⊤ =
⎡
⎢⎢⎣
......
u1 · · · un
......
⎤
⎥⎥⎦
⎡
⎢⎣σ1
. . .σn
⎤
⎥⎦
⎡
⎢⎣· · · v⊤1 · · ·
...· · · v⊤n · · ·
⎤
⎥⎦
Hf : �2 � �2
g �� Hfg
(Hfg)(x) =�
y���
f(xy)g(y)
Key Idea: see Hankel matrix as an operator in a Hilbert spaceTheorem:
Hf with f rational admitsan SVD iff �f�2 < �
[Balle, Panangaden, Precup 2015, 201?]
![Page 29: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/29.jpg)
The Singular Value AutomatonFrom Correspondence Theorem:
If Hf admits an SVD Hf = UDV�, then there exists a WAA for f inducing P = UD1/2 and S = D1/2V�
[Balle, Panangaden, Precup 2015]
![Page 30: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/30.jpg)
The Singular Value AutomatonFrom Correspondence Theorem:
If Hf admits an SVD Hf = UDV�, then there exists a WAA for f inducing P = UD1/2 and S = D1/2V�
singular value automaton
[Balle, Panangaden, Precup 2015]
![Page 31: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/31.jpg)
The Singular Value AutomatonFrom Correspondence Theorem:
If Hf admits an SVD Hf = UDV�, then there exists a WAA for f inducing P = UD1/2 and S = D1/2V�
WA computing sing. vect. can be obtained from SVA:
Ui = ⟨α, ei/√σi, {Ta}⟩
Vi = ⟨ei/√σi,β, {Ta}⟩
fUi= ui
fVi= vi
A = ��, �, {Ta}a���
singular value automaton
[Balle, Panangaden, Precup 2015]
![Page 32: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/32.jpg)
Consequences of SVA
1. For every f � �2Rat the SVA provides a “unique”canonical WA representation
2. The vector subspace �2Rat � R��is closed under
taking left/right singular vectors of Hankel matrices
![Page 33: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/33.jpg)
Consequences of SVA
1. For every f � �2Rat the SVA provides a “unique”canonical WA representation
2. The vector subspace �2Rat � R��is closed under
taking left/right singular vectors of Hankel matrices
![Page 34: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/34.jpg)
The Plan
1. Hankel Matrices and Singular Value Automata (SVA)
2. Algorithms for Computing SVA
3. WA Approximation via SVA Truncation
![Page 35: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/35.jpg)
Two Important Matrices
Reachability Gramian: GP = P> · P
Observability Gramian: GS = S · S>
![Page 36: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/36.jpg)
Two Important Matrices
Reachability Gramian: GP = P> · P
Observability Gramian: GS = S · S>
Theorem:Assuming A is minimal, GP and GS are well-defined iff �fA�2 < �
[Balle, Panangaden, Precup 2015, 201?]
![Page 37: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/37.jpg)
Two Important Matrices
Reachability Gramian: GP = P> · P
Observability Gramian: GS = S · S>
P = UD1/2 ) GP = D
S = D1/2V> ) GS = D
For SVA
![Page 38: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/38.jpg)
Two Important Matrices
Reachability Gramian: GP = P> · P
Observability Gramian: GS = S · S>
GQP = Q>GPQ
GQS = Q-1GSQ
->
AQFor
P = UD1/2 ) GP = D
S = D1/2V> ) GS = D
For SVA
![Page 39: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/39.jpg)
Computing the SVAGiven minimal WA A computing f � �2Rat do:
1. Compute Gramians GP and GS
2. Diagonalize M = GSGP
(i.e. find Q s.t. Q�1MQ = D2)
3. Obtain SVA as AQ
![Page 40: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/40.jpg)
Computing the SVA
note:Step 2 is a finite eigenvalue problem :-)
Step 1 involves products of infinite matrices :-o
Given minimal WA A computing f � �2Rat do:
1. Compute Gramians GP and GS
2. Diagonalize M = GSGP
(i.e. find Q s.t. Q�1MQ = D2)
3. Obtain SVA as AQ
![Page 41: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/41.jpg)
Computing the Gramians
Fixed-point Equations for Gramians
GS = β · β⊤ +∑
a∈Σ
Ta ·GS · T⊤a
GP = α⊤ · α+∑
a∈Σ
T⊤a ·GP · Ta
![Page 42: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/42.jpg)
Computing the Gramians
Closed-form Algorithm
Total time O(n6 + |�|n4)vec(GS) =
�
I ��
a��
Ta � Ta
��1
(� � �)
Fixed-point Equations for Gramians
GS = β · β⊤ +∑
a∈Σ
Ta ·GS · T⊤a
GP = α⊤ · α+∑
a∈Σ
T⊤a ·GP · Ta
![Page 43: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/43.jpg)
Computing the Gramians
Closed-form Algorithm
Total time O(n6 + |�|n4)vec(GS) =
�
I ��
a��
Ta � Ta
��1
(� � �)
Fixed-point Equations for Gramians
GS = β · β⊤ +∑
a∈Σ
Ta ·GS · T⊤a
GP = α⊤ · α+∑
a∈Σ
T⊤a ·GP · Ta
note: there is also an iterative algorithm with O(|�|n2.4) flops per iteration
![Page 44: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/44.jpg)
The Plan
1. Hankel Matrices and Singular Value Automata (SVA)
2. Algorithms for Computing SVA
3. WA Approximation via SVA Truncation
![Page 45: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/45.jpg)
Approximation (Attempt 1)⎡
⎢⎢⎣
......
u1 · · · un
......
⎤
⎥⎥⎦
⎡
⎢⎣σ1
. . .σn
⎤
⎥⎦
⎡
⎢⎣· · · v⊤1 · · ·
...· · · v⊤n · · ·
⎤
⎥⎦ ≈
⎡
⎢⎢⎣
......
u1 · · · uk
......
⎤
⎥⎥⎦
⎡
⎢⎣σ1
. . .σk
⎤
⎥⎦
⎡
⎢⎣· · · v⊤1 · · ·
...· · · vk
⊤ · · ·
⎤
⎥⎦
Hf H
![Page 46: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/46.jpg)
Approximation (Attempt 1)⎡
⎢⎢⎣
......
u1 · · · un
......
⎤
⎥⎥⎦
⎡
⎢⎣σ1
. . .σn
⎤
⎥⎦
⎡
⎢⎣· · · v⊤1 · · ·
...· · · v⊤n · · ·
⎤
⎥⎦ ≈
⎡
⎢⎢⎣
......
u1 · · · uk
......
⎤
⎥⎥⎦
⎡
⎢⎣σ1
. . .σk
⎤
⎥⎦
⎡
⎢⎣· · · v⊤1 · · ·
...· · · vk
⊤ · · ·
⎤
⎥⎦
Hf H best rank k approximation
![Page 47: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/47.jpg)
Claim:
If f : �� � R is such that Hf = H, then rank(f) = k
and �f � f�2 � �Hf � H�op = �k+1
Approximation (Attempt 1)⎡
⎢⎢⎣
......
u1 · · · un
......
⎤
⎥⎥⎦
⎡
⎢⎣σ1
. . .σn
⎤
⎥⎦
⎡
⎢⎣· · · v⊤1 · · ·
...· · · v⊤n · · ·
⎤
⎥⎦ ≈
⎡
⎢⎢⎣
......
u1 · · · uk
......
⎤
⎥⎥⎦
⎡
⎢⎣σ1
. . .σk
⎤
⎥⎦
⎡
⎢⎣· · · v⊤1 · · ·
...· · · vk
⊤ · · ·
⎤
⎥⎦
Hf H best rank k approximation
![Page 48: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/48.jpg)
Claim:
If f : �� � R is such that Hf = H, then rank(f) = k
and �f � f�2 � �Hf � H�op = �k+1
Approximation (Attempt 1)⎡
⎢⎢⎣
......
u1 · · · un
......
⎤
⎥⎥⎦
⎡
⎢⎣σ1
. . .σn
⎤
⎥⎦
⎡
⎢⎣· · · v⊤1 · · ·
...· · · v⊤n · · ·
⎤
⎥⎦ ≈
⎡
⎢⎢⎣
......
u1 · · · uk
......
⎤
⎥⎥⎦
⎡
⎢⎣σ1
. . .σk
⎤
⎥⎦
⎡
⎢⎣· · · v⊤1 · · ·
...· · · vk
⊤ · · ·
⎤
⎥⎦
Hf H best rank k approximation
not Hankel!
![Page 49: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/49.jpg)
Approximation (Attempt 2)
Hf =
⎡
⎢⎢⎣
...f...
⎤
⎥⎥⎦ =
⎡
⎢⎢⎣
......√
σ1u1 · · · √σnun
......
⎤
⎥⎥⎦
⎡
⎢⎣β1...
βn
⎤
⎥⎦
UD1/2 D1/2V⊤
![Page 50: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/50.jpg)
Approximation (Attempt 2)
f =n∑
i=1
βi√σiuiHf =
⎡
⎢⎢⎣
...f...
⎤
⎥⎥⎦ =
⎡
⎢⎢⎣
......√
σ1u1 · · · √σnun
......
⎤
⎥⎥⎦
⎡
⎢⎣β1...
βn
⎤
⎥⎦
UD1/2 D1/2V⊤
![Page 51: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/51.jpg)
Claim:
Taking f =�k
i=1 �i�
�iui we have a rational functionsum of k orthogonal rational functions and �f � f�2
2 =�ni=k+1 �2
i�i � �ni=k+1 �2
i
Approximation (Attempt 2)
f =n∑
i=1
βi√σiuiHf =
⎡
⎢⎢⎣
...f...
⎤
⎥⎥⎦ =
⎡
⎢⎢⎣
......√
σ1u1 · · · √σnun
......
⎤
⎥⎥⎦
⎡
⎢⎣β1...
βn
⎤
⎥⎦
UD1/2 D1/2V⊤
![Page 52: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/52.jpg)
Claim:
Taking f =�k
i=1 �i�
�iui we have a rational functionsum of k orthogonal rational functions and �f � f�2
2 =�ni=k+1 �2
i�i � �ni=k+1 �2
i
Approximation (Attempt 2)
f =n∑
i=1
βi√σiui
has rank n, but want k!
Hf =
⎡
⎢⎢⎣
...f...
⎤
⎥⎥⎦ =
⎡
⎢⎢⎣
......√
σ1u1 · · · √σnun
......
⎤
⎥⎥⎦
⎡
⎢⎣β1...
βn
⎤
⎥⎦
UD1/2 D1/2V⊤
![Page 53: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/53.jpg)
SVA Truncation
q1 q2
q3 q4
Keep
Remove
![Page 54: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/54.jpg)
SVA Truncation
q1 q2
q3 q4
Keep
Remove
corresponding to k largest singular values
![Page 55: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/55.jpg)
SVA Truncation
q1 q2
q3 q4
Keep
Remove
corresponding to k largest singular values
A� =
2
664
• • • •• • • •• • • •• • • •
3
775
keep to keep
remove to removeremove to keep
keep to remove
Tσ
![Page 56: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/56.jpg)
Error in SVA TruncationOriginal Truncated
A� =
2
664
• • • •• • • •• • • •• • • •
3
775 A� =
�• •• •
�Tσ Tσ
A, f, n states A, f, k states
![Page 57: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/57.jpg)
Error in SVA TruncationOriginal Truncated
A� =
2
664
• • • •• • • •• • • •• • • •
3
775 A� =
�• •• •
�Tσ Tσ
A, f, n states A, f, k states
Error Bounds:
�f � f�2 = �
��
�
Cf(�k+1 + · · · + �n)1/4
(�2k+1 + · · · + �2
n)1/2
[Balle, Panangaden, Precup 2015, 201?, Kulesza, Jiang, Singh 2015]
![Page 58: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/58.jpg)
Conclusion
![Page 59: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/59.jpg)
Conclusion
• Tools from functional analysis must play a role in a theory of automata approximation
![Page 60: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/60.jpg)
Conclusion
• Tools from functional analysis must play a role in a theory of automata approximation
• Approximation theories can lead to useful algorithms for large-scale problems
![Page 61: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/61.jpg)
Conclusion
• Tools from functional analysis must play a role in a theory of automata approximation
• Approximation theories can lead to useful algorithms for large-scale problems
• Change of basis in WA doesn’t change the function but yields automata with different properties (eg. for truncation)
![Page 62: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/62.jpg)
Open Problems
![Page 63: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/63.jpg)
Open Problems• How can this be extended to other types of automata, state
machines, co-algebras, etc?
![Page 64: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/64.jpg)
Open Problems• How can this be extended to other types of automata, state
machines, co-algebras, etc?
• How important is it that WA are closed under reversal for the error analysis? [see Rabusseau, Balle, Cohen 2016]
![Page 65: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/65.jpg)
Open Problems• How can this be extended to other types of automata, state
machines, co-algebras, etc?
• How important is it that WA are closed under reversal for the error analysis? [see Rabusseau, Balle, Cohen 2016]
• Can the AAK theorems in control theory be generalised to WA?
![Page 66: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/66.jpg)
Open Problems• How can this be extended to other types of automata, state
machines, co-algebras, etc?
• How important is it that WA are closed under reversal for the error analysis? [see Rabusseau, Balle, Cohen 2016]
• Can the AAK theorems in control theory be generalised to WA?
• Can we obtain efficient approximation algorithms with extra constraints (eg. sparsity, positivity)?
![Page 67: (Co)-Algebraic and Analytical Aspects of Weighted Automata ... · Minimisation vs Approximation • Minimisation is naturally related to (co-)algebraic properties • Approximation](https://reader033.vdocuments.mx/reader033/viewer/2022053109/607def1c14ebf804ab37ae72/html5/thumbnails/67.jpg)
(Co)-Algebraic and Analytical Aspects of Weighted Automata Minimisation and Equivalence
(Part 2)
Borja Balle
[CMCS Tutorial, Apr 2 2016]