G2ELAB

Model Order Reduction Partial Report

Mateus Antunes Oliveira Leite

24/04/2015

Partial report containing the theory of model order reduction

1. Topics in Linear Algebra

In order to introduce the concepts of model order reduction in a self-contained manner, some

important topics in linear algebra are presented.

Vector basis

Any vector in a ℝ𝑛 space can be represented as a linear combination of n linearly independent base

vectors. In Equation (1), 𝐛i represents a base vector of ℝ𝑛 with index i and 𝐯 is the vector being

composed.

𝐯 = 𝛼1𝐛𝟏 + 𝛼2𝐛𝟐 + ⋯ + 𝛼𝑛𝐛𝐧 (1)

In matrix notation, this relation can be expressed as in Equation (2). The columns of matrix 𝐁 are the

base vectors and the vector 𝛂 contains the coefficients of each of the base vectors. The vector 𝐯 is

said to be in the column space of B.

𝐯 = 𝐁𝛂 (2)

It must be pointed out that vectors are geometric objects and thus are independent of the basis one

chooses the represent them. The vectors 𝐯 and 𝛂 can be regarded as the same geometric vector but

in different basis. Therefore, Equation (2) can be regarded as change of basis. Once the columns of 𝐁

are linearly independent, the matrix is not singular and its inverse exists. This permits to express the

inverse mapping as represented in Equation (3).

𝛂 = 𝐁−𝟏𝐯 (3)

Projections

In this section, a simple 3-dimensional example of projection is presented. However, the derived

results are general and can be applied directly to an arbitrary number of dimensions.

Figure 1 - Projection into a 2-dimensional space

Ω

v

p

e

b1

b2

The Figure 1 represents a projection of the vector 𝐯 into the subspace . The resulting or projected

vector is 𝐩 and 𝐞 is the different among these two quantities. The subspace represented by the

matrix 𝛀 can be constructed by arranging the base vectors 𝐛𝟏 and 𝐛𝟐 as its columns. Thus, any vector

lying in this subspace can be represented by a relation similar to Equation (2). Therefore, the relation

among the vectors v, p and e is given by Equation (4).

𝐯 = 𝐩 + 𝐞 (4)

The fact that 𝐩 is in the column span of 𝛀 allows writing equation (4) as in Equation (5).

𝐯 = 𝛀𝛂 + 𝐞 (5)

To determine the vector , one should choose a subspace 𝚿 orthogonal to e, obtaining an explicit

expression for , as shown in Equation (6).

𝛂 = (𝚿T𝛀)−1

𝚿T𝐯 (6)

Therefore, the projected vector can be obtained by Equation (7).

𝐩 = 𝛀(𝚿T𝛀)−1

𝚿T𝐯 (7)

The linear operator representing the oblique projection of the components of 𝐯 that are orthogonal

to 𝚿 into 𝛀 can be extracted directly from this relation as expressed in Equation (8).

𝚷Ω,Ψ = 𝛀(𝚿T𝛀)−1

𝚿T (8)

If both the subspaces 𝛀 and 𝚿 are the same, this projection is called an orthogonal projection. This

kind of projection has a very important relation to the following optimization problem: find a vector

𝐩 lying in the subspace 𝛀 that best approximates the vector v. In the current context, the best

approximation is obtained when the square of the length of the error is minimized. Using the law of

cosines, the error vector represented in Figure 1 is given by Equation (9).

|𝐞|2 = 𝐯T𝐯 − 2𝐯T𝐩 + 𝐩T𝐩 (9)

Using the fact that 𝐩 is in the column space of , the above relation can be written as in Equation

(10).

|𝐞|2 = 𝐯T𝐯 − 2𝐯T𝛀𝛂 + (𝛀𝛂)T𝛀𝜶 (10)

Taking the gradient of the right hand side in relation to α and imposing the condition of it being zero,

one can isolate the vector . This process is shown in (11). See the appendix for the used math

property.

𝛁(𝐯T𝐯 − 2𝐯T𝛀𝛂 + (𝛀𝛂)T𝛀𝜶) = 𝟎

𝛁(−2𝐯T𝛀𝛂) + 𝛁[(𝛀𝛂)T𝛀𝜶] = 𝟎

−2[𝛁𝐯T𝛀𝛂 + 𝛁(𝛀𝛂)T𝐯] + 2𝛁(𝛀𝛂)T𝛀𝜶 = 𝟎

𝛀T𝐯 = 𝛀T𝛀𝜶

𝜶 = (𝛀T𝛀)−1

𝛀T𝐯

(11)

Therefore the vector that minimizes the square of the modulus of the error is given by Equation (12).

𝐩 = 𝛀(𝛀T𝛀)−1

𝛀T𝐯 (12)

Comparing Equations (8) and (12), one can conclude that the optimal approximation vector is just an

orthogonal projection into the subspace .

A very important application of this fact that will be largely used in model order reduction is that it

can be used to approximately solve overdetermined systems. As an example, imagine that one wants

to solve the linear system of Equation (13) with 𝐀 ∈ ℝ𝑛,𝑚, 𝐱 ∈ ℝ𝑚 and 𝐛 ∈ ℝ𝑛 with n > m.

𝐀𝐱 = 𝐛 (13)

This problem is overdetermined and can be interpreted geometrically as trying to find the

representation of the vector 𝐛 using the columns of 𝐀 as the base vectors. It is very unlikely that this

task is possible, once that the number of base vector to span the totality of ℝ𝑛 is equal to n but only

m base vectors are available. Therefore, the available basis represents a subspace of dimension m

embedded on a higher n-dimensional space. One approach to solve this problem in an approximated

fashion is to project the vector 𝐛 into the column span of A. This results in shown in Equation (14).

𝐀𝐱 = 𝐀(𝐀T𝐀)−1

𝐀T𝐛 (14)

One can multiply both sides of the above equation by 𝐀T to obtain Equation (15).

𝐀T𝐀𝐱 = 𝐀T𝐛 (15)

Therefore, it is possible to conclude that to obtain the closest approximation to Equation (13), in the

sense of least squares, it is only necessary to multiply the system of equations by the transpose of

the coefficient matrix.

Similarity Transformation

Matrices may be used to represent linear operators; an example of this is the projection operator. If

two bases for the same linear space are related by a change of coordinates as shown by Equation

(16), it is natural to wonder how to transform the operator between coordinate systems. One may

start with Equation (17), that represents the application of a transformation A to a vector x.

𝐱 = 𝐓𝐲 (16)

𝐀𝐱 = �̅� (17)

Using Equation (16) into (17), the relation for y and the transformed operator B may be obtained.

This is shown in Equation (18).

𝐁𝐲 = 𝐓−1𝐀𝐓𝐲 = �̅� (18)

This type of relation has a very important property: the eigenvalues of A are the same as B’s. This can

be easy shown by the development in (19).

|𝐓−1𝐀𝐓 − 𝜆𝐈| = |𝐓−1𝐀𝐓 − 𝜆𝐓−1𝐓| = |𝐓−1||𝐀 − λ𝐈||𝐓| = |𝐀 − λ𝐈| (19)

In a similar fashion, one may wonder of what are the relations of the eigenvalues of a transformation

of the kind indicated in Equation (20).

𝐀𝐫 = 𝐏T𝐀𝐏 (20)

For this analysis the focus is not to show that the eigenvalues are the same, which they are not, but

to demonstrate that their signs are preserved. This is very important because, as will be explained

latter in this document, the sign of the eigenvalues determines if the system is stable or not.

If A is symmetric, and the system is stable, then it can be decomposed as written in Equation (21).

𝑨 = −𝑩𝑻𝑩 (21)

This allows to prove that the reduced matrix is negative definite as shown in (22). For a general A

matrix, there is no guarantee that the resulting reduced system will be stable.

𝐱𝐓𝐀𝐫𝐱 = 𝐱𝐓𝐏T𝐀𝐏𝐱 = −𝐱𝐓𝐏𝐓𝑩𝑻𝑩𝑷𝐱 = −(𝑩𝑷𝒙)𝑻(𝑩𝑷𝒙) = −‖𝑩𝑷𝒙‖ ≤ 𝟎 (22)

Singular Value Decomposition

Any matrix 𝐀 ∈ ℝm,n may be decomposed as a product of two orthonormal and a diagonal matrix as

shown in Equation (23).

𝐀 = 𝐔𝚺𝐕∗ (23)

To be able to determine these matrices, one may write the product of Equation (24).

𝐀𝐀∗ = 𝐔𝚺𝐕∗𝐕𝚺∗𝐔∗ = 𝐔𝚺𝚺∗𝐔∗ (24)

This leads to conclude that the matrix U is build using the eigenvectors of the matrix AA*, and each

of the diagonal entries of Σ is the square root of the not null eigenvalues of AA*. A similar argument

shows that the matrix V is composed by the eigenvectors of the matrix A*A. The first n columns of

matrix U are an orthonormal basis for the range of A.

This decomposition allows writing any matrix as a sum of products of rank 1. This is written in

Equation (25). It is possible to show that truncating this series at position r leads to the best rank r

approximation of matrix A.

𝐀 = σ1𝐮𝟏𝐯𝟏∗ + σ1𝐮𝟏𝐯𝟏

∗ + ⋯ + σr𝐮𝐫𝐯𝐫∗ (25)

2. Model Order Reduction by Projection

State Space Representation

The representation of time invariant linear systems used in this document is given by Equations (26)

and (27). In these equations: 𝐄 ∈ ℝ𝑛,𝑛, 𝐀 ∈ ℝ𝑛,𝑛, 𝐁 ∈ ℝ𝑛,𝑟, 𝐂 ∈ ℝ𝑠,𝑛, 𝐃 ∈ ℝ𝑠,𝑟, 𝐱 ∈ ℝ𝑛, 𝐮 ∈ ℝ𝑟,

𝐲 ∈ ℝs. The vector x is the state vector, y is the output vector and u is the input vector.

𝐄�̇� = 𝐀𝐱 + 𝐁𝐮 (26)

It must be pointed out that Equations (26) and (27) are not coupled.

𝐲 = 𝐂𝐱 + 𝐃𝐮 (27)

For now, the matrix D is considered of all zeros. If this is not the case in a particular application, the

reasoning presented below can be easily adapted.

If the matrix E is not singular it is possible to write the system in a simpler form as shown in (28).

�̇� = 𝐄−1𝐀𝐱 + 𝐄−1𝐁𝐮 = 𝐀′𝐱 + 𝐁′𝐮 (28)

Many textbooks present this formulation as the standard form for the state space system. However,

problems arise when E is singular. This indicates the presence of algebraic states that have to be

threated explicitly. The first step is to write Equation (26) as in Equation (29).

[𝐄11 𝟎𝟎 𝟎

] [�̇�1

�̇�2] = [

𝐀11 𝐀12

𝐀21 𝐀22] [

𝐲1

𝐲2] + [

𝐁1

𝐁2] 𝐮 (29)

To obtain this form, permutations of the lines and columns of the E matrix should be followed by

equivalent permutations of A and B. To achieve this, one may use a permutation matrix P to create

the transformation of Equation (30).

𝐱 = 𝐏𝐲 (30)

Accordingly, the transformed system is given by Equation (31).

𝐏T𝐄𝐏𝐲 = 𝐏T𝐀𝐏𝐲 + 𝐏T𝐁𝐮 (31)

In this stage it is possible to apply a Kron reduction to Equation (29) to obtain (32).

𝐄11�̇�1 = (𝐀11 − 𝐀12𝐀22−1𝐀21)𝐲𝟏 + (𝐁1 − 𝐀12𝐀22

−1𝐁2)𝐮 (32)

This representation is a reduced state space equation that allows direct reduction to standard form.

Thus, writing the system as in Equation (26) or as in Equation (28) is interchangeable and both

formulations may be used when they are more convenient.

Observability and Reachability

There are two quantities that are very important to characterize the system. Suppose a system

written as in Equation (28), the response to an arbitrary input signal is given by Equation (33).

𝐱(𝑡) = 𝑒𝐀(𝑡−𝑡0)𝐱(𝑡0) + ∫ 𝑒𝐀(𝑡−𝜏)𝐁𝐮(τ)dτ𝑡

𝑡0

(33)

If in the beginning of the simulations the system is found with zero initial conditions, it is a natural

question to ask which is the input signal u(t) that may be used to drive the system to a given state.

There may be an infinite number of signals that can accomplish this task. To reduce the number of

choices, one may want that this signal is optimal in the sense that it is capable to drive the system to

the desired state using the minimum amount of energy. For simplicity, but without any loss of

generality, the initial time is set to zero. The equation that describes the state evolution of this

system is given by (34).

x(t) = ∫ 𝑒𝐀(t−𝜏)𝐁𝐮(τ)dτ𝑡

0

(34)

If one chooses the input function as in Equation (35), the above identity is satisfied in the least

amount of energy [1]. The matrix 𝒫 which is called the controllability Gramian is defined by

Equation (36). Some authors use t = ∞ to define this quantity and they call it the infinite

Gramian.

𝐮(𝜏) = 𝐁∗𝑒𝐀∗(𝑡−𝜏)𝓟−1(𝑡)𝐱(𝑡) (35)

𝓟 = ∫ 𝑒𝐀(t−τ)𝐁𝐁∗𝑒𝐀∗(𝑡−𝜏)𝑑𝜏𝑡

0

= ∫ 𝑒𝐀𝜉𝐁𝐁∗𝑒𝐀∗𝜉𝑑𝜉𝑡

0

(36)

The energy in this signal can by written as in (37). Its physical interpretation leads to the conclusion

that the controllability Gramian measures how hard is to achieve a certain state. In the framework of

model order reduction, states that are difficult to reach are good candidates for truncation.

Ec = ∫ 𝐮(𝜏)∗𝐮(𝜏)𝑑𝜏 = 𝐱0∗ 𝓟−1𝐱0

𝑡

0

(37)

The above discussion involved solely the states of the system and ignored its output. A dual concept

called observability can be derived if one analyses the amount of energy that a certain state delivers

to the output if no input signal is present. The system response to this scenario is given by Equation

(38).

𝐱0 = 𝐂𝑒𝐀𝑡𝐱𝟎 (38)

The energy of this signal can be calculates with aid of Equation (37) and the result is given by

Equation (39) which is also the definition of the observability Gramian.

Eo = 𝐱0∗ ∫ 𝑒𝐀∗𝝉𝐂∗𝐂𝑒𝐀𝜏𝑑𝜏

𝑡

0

𝐱0 = 𝐱𝟎∗ 𝓠𝐱0 (39)

Controllability and Observability are dual concepts and their relation is shown in Table 1.

Table 1 – Duality relation among controllability and observability

Controllability Observability

A A* B C*

Reduction by Projection

If it is assumed that n is a large number, the burden of solving Equation (26) is very high or even

prohibitive. Therefore, a reduction of the system may be used to allow a faster solution with an

acceptable loss of accuracy. As presented in Equation (2), any two bases for ℝn can be related by a

transformation matrix. The transformation matrix can be partitioned into two smaller matrices,

allowing one to obtain Equation (40). V contains q base vector and U the last n – q.

𝐱 = 𝐕𝐱𝐫 + 𝐔𝐱𝐬 (40)

Then, Equation (26) may be written as in (41).

𝐄𝐕�̇�𝐫 = 𝐀𝐕𝐱𝐫 + 𝐁𝐮 + 𝐀𝐔𝐱𝐬 − 𝐄𝐔�̇�𝐬 (41)

One may choose the basis in a way that the vectors of V are more important in representing the

dynamics of the system than the vectors in U. Thus, the last two terms of the right hand side can be

interpreted as an error factor.

𝐄𝐕�̇�𝐫 = 𝐀𝐕𝐱𝐫 + 𝐁𝐮 + 𝝐 (42)

The above system in xr is overdetermined. The same process used in Equations (13), (14) and (15) can

be used to allow the solution of this equation. This is shown in (43).

𝐖T𝐄𝐕�̇�𝐫 = 𝐖T𝐀𝐕𝐱𝐫 + 𝐖T𝐁𝐮 + 𝐖T𝝐 (43)

Until this point, the relation is exact and does not lead to any computational improvement. However,

if the last term of the right rand side of (43) is neglected, one can write the equations for the reduced

system. This is shown in Equations (44) and (45).

𝐖T𝐄𝐕�̇�𝐫 = 𝐖T𝐀𝐕𝐱𝐫 + 𝐖T𝐁𝐮 (44)

𝐲 = 𝐂𝐕𝐱𝐫 (45)

Under the light of projection matrices, the above equations represent the projection of the system

into a subspace spanned by V and orthogonal to W. The above set of equations allows the solution of

an approximation to the original system but with a reduced order.

For simplicity, a relation between the original system and reduced system matrices can be made. This

allows writing the reduced system in standard form. This relation is presented in Table 2.

Table 2 - Reduced System Coefficient Matrices

Original System Reduced System

𝐄 𝐖T𝐄𝐕 𝐀 𝐖T𝐀𝐕 𝐁 𝐖T𝐁 𝐂 𝐂𝐕

The reduced system can be written as in Equations (46) and (47).

𝐄𝐫�̇�𝐫 = 𝐀𝐫𝐱𝐫 + 𝐁𝐫𝐮 (46)

𝐲 = 𝐂𝐫𝐱𝐫 (47)

To deal with initial conditions, one could use the projection of the initial condition vector into the

span of the reduced basis.

Invariance of the Transfer Function

In the frequency domain, the transfer function of the system is given by Equation (48). This is

valid for the full and reduced system as well if one keeps in mind the relations of Table 2.

𝐇 = 𝐂(s𝐄 − 𝐀)−1𝐁 (48)

If instead of using matrices V and W for the projection, one utilizes different matrices but with the

same column span, the resulting transfer function is unchanged. If the matrices K and L are

nonsingular, the matrices V’ and W’, defined in Equations (49) and (50), have the same column span

of V and W, respectively.

𝐕′ = 𝐕𝐊 (49)

𝐖′ = 𝐖𝐋 (50)

Substituting these matrices into Equation (48), Equation (51) is obtained.

𝐇 = 𝐂𝐕′𝐊−1(s(𝐖′𝐋−1)T𝐄𝐕′𝐊−1 − (𝐖′𝐋−1)T𝐀𝐕′𝐊−1)−1

(𝐖𝐋−1)T𝐁 (51)

Developing the above expression, the expression to the transformed reduced system is found to be

identical to the original reduced system. Therefore, it may be concluded that the transfer function is

invariant under a change of base. The important aspects of matrices V and W are their column span.

Balanced Truncation

The observability and reachability of each state is dependent of the system realization. If one applies

a base change represented by the matrix T, the transformed Gramians are given by (52).

�̃� = 𝑻−𝟏𝓟𝑻−∗ �̃� = 𝑻∗𝓠𝑻

(52)

It is possible to choose the matrix T such that both Gramians are equal and diagonal. This can be

achieved by a singular value decomposition of the product (53).

𝓟𝓠 = 𝑻𝚺𝟐𝑻−𝟏 (53)

In order to reduce the system, the states that have small associated diagonal values. The problem of

this method is the burden to calculate the Gramians that is of the order of n3 [2]. The standard way

to do this is to solve the Lyapunov equations given by (54). The verification of this fact can be made

by direct substitution of the definition of the Gramians.

𝑨𝓟 + 𝓟𝑨∗ + 𝑩𝑩∗ = 𝟎𝑨∗𝓠 + 𝓠𝑨 + 𝑪∗𝑪 = 𝟎

(54)

Proper Orthogonal Decomposition

The development of this section is largely inspired in [3]. The interested reader may look at this work

for further discussion.

The proper orthogonal decomposition aims at finding a projection operator of fixed rank (Π) that

minimizes the quadratic error created by the simulation of a lower order system. For the case of

continuous time, the quadratic error is given by Equation (55). We already know that one may

truncate Equation (25) to solve directly this problem. However, the following development raises a

lot of understanding of the problem structure.

𝜖 = ∫ ‖𝐱(𝑡) − 𝚷𝐱(𝑡)‖2𝑑𝑡𝑻

𝟎

(55)

The integrand is the projection of the vector x into the space that is orthogonal to the projected one.

As any vector can be written as the sum of the projection into a subspace and the projection to the

orthogonal subspace, minimizing the error is equal to maximize the projection into the orthogonal

space. This is shown in Equation (56).

𝑀 = ∫ ‖𝚷𝐱(𝑡)‖2𝑑𝑡𝑻

𝟎

(56)

Using Equation (12) and imposing the orthonormality of the base, the development in (57) is

possible. In this equation, tr represents the trace.

𝑀 = ∫ ‖𝚷𝐱(𝑡)‖2𝑑𝑡𝑻

𝟎

= ∫ ‖𝛀T𝐱(𝑡)‖2

𝑑𝑡𝑻

𝟎

= 𝑡𝑟 (𝛀𝑻 ∫ 𝐱(𝑡)𝐱(𝑡)𝑇 𝑑𝑡𝑻

𝟎

𝛀) (57)

This motivates the definition of the POD Kernel given by Equation (58).

𝐊 = ∫ 𝐱(𝑡) 𝐱(𝑡)𝑇 𝑑𝑡𝑻

𝟎

(58)

This development can be cast into an optimization problem whose Lagrangian is given by (59).

𝓛(𝑽, 𝜶) = 𝒕𝒓(𝛀𝑻𝑲𝛀) − ∑ 𝜶𝒊,𝒋(𝒗𝒊𝑻𝒗𝒋 − 𝜹𝒊𝒋)

𝒊,𝒋

(59)

It is possible to cast the orthogonality restriction as in Equation (60) if one defines the matrix A as in

(61).

𝓛(𝑽, 𝜶) = 𝑡𝑟(𝛀𝑻𝑲𝛀) − 𝑡𝑟(𝑨𝑻𝑽𝑻𝑽) + 𝑡𝑟(𝑨) (60)

𝑨𝒊𝒊 = 𝜶𝒊𝒊 , 𝑨𝒊𝒋 =𝟏

𝟐𝜶𝒊𝒋

(61)

Using the properties in the appendix, one may calculate the first order optimality condition, in

relation to Ω, to obtain Equation (62).

𝐊𝐕 = 𝐕𝐀 (62)

This condition implies that V must span a subspace that is invariant under the application of the

linear operator K. The first order optimality condition for the α values leads to the obvious restriction

given by Equation (63).

𝐕T𝐕 = ℑ (63)

Both of these conditions are satisfied if V is built with the eigenvectors of K. To calculate K in a digital

computer, a discretization is necessary. Equation shows a way to approximate the kernel. The S

matrix is defined in (65).

𝐊 = ∫ 𝑥(𝑡) 𝑥(𝑡)𝑇 𝑑𝑡𝑻

𝟎

= ∑ 𝒙(𝒕𝒊)𝒙(𝒕𝒊)𝑻𝚫𝒕𝒊

𝒊

= 𝑺𝑺𝑻 (64)

𝐒 = [𝒙𝟎 √𝚫𝒕𝟎 𝒙𝟏 √𝚫𝒕𝟏 ⋯ 𝒙𝒏𝒔 √𝚫𝒕𝒏𝒔] (65)

As the kernel is an n-by-n matrix, the burden to compute the eigenvectors may be very high.

However it is possible to avoid this by the application of the singular value decomposition of S. This is

shown in Equation (66).

𝑆𝑆𝑇 = (𝑈Σ𝑉𝑇)(𝑈Σ𝑉𝑇)𝑇 = 𝑈ΣΣ𝑇𝑈𝑇 (66)

Therefore, the eigenvectors that we are looking for are the columns of the U matrix of the singular

value decomposition of the kernel. To compute U in an efficient way, the ‘economic’ version of the

SVD may be used. Mathematically this is equivalent to compute V by the eigenvalue decomposition

of the product STS, that has a size much smaller than the Kernel, and then using Equation (67) to get

only some eigenvectors. This does not alter the problem once that the column span of the product is

unchanged by the multiplication by Σ.

𝑈Σ = 𝑆𝑆𝑇𝑉 (67)

It must be pointed out that this is only the mathematical description of the solution. In a computer

implementation this procedure may be replaced by more efficient ones.

In this method, only the system states are taken into account. No information is obtained from the

system output. This can be easily solved by using a second system that satisfies the relations in Table

1, called the dual system. Using information from these two systems allows to approximate the

balanced truncation method in a less expensive way.

Moment Matching

The transfer function in Equation (48) can be rewritten as in Equation (68).

𝐇 = −𝐂(s𝐀−1𝐄 − 𝐈)−1𝐀−1𝐁 (68)

The central term can be expanded to obtain a series representation of the transfer function. This is

shown in (69).

𝐇 = − ∑ 𝐂(𝐀−1𝐄)𝑗𝐀−1𝐁s𝑗

∞

𝑗=0

(69)

The negative of these terms in s are called the moments of the transfer function. These moments are

centered at the zero frequency. To obtain the moments for other frequencies, the transfer function

can be written as in Equation (85).

𝐇 = 𝐂((𝑠 − σ)𝐄 − (𝐀 − σ𝐄))−1

𝐁 (70)

Direct comparison of Equations (48) and (70) allows determining the equivalences pointed out in

Table 3. Table 3 - Equivalence for decentered moments

Centered at Zero Centered at σ

𝑠 𝑠 − 𝜎 𝐀 𝐀 − 𝜎𝐄

Using these relations of can directly write the moments for any frequency. This is written in Equation

(71). This relation can be used to extend other results obtained to the zero centered expression to an

arbitrary placed frequency.

𝐇 = − ∑ 𝐂[(𝐀 − 𝜎𝐄)−1𝐄]𝑗(𝐀 − 𝜎𝐄)−1𝐁(s − σ)j

∞

𝑗=0

(71)

The moment matching technique aims at choosing the columns of V and W in a way that some of the

moments of the transfer function are exactly matched. The first moment of the reduced model is

given by Equation (72).

𝐇 = 𝐂𝐕(𝐖T𝐀𝐕)−𝟏

𝐖T𝐁 (72)

If A-1B is in the column span of V, there exists a vector r0 as shown in Equation (88).

𝐕𝐫0 = 𝐀−1𝐁 (73)

Equation (72) can be written as in (74). Therefore, the zeroth moment of the reduced system is equal

of the zeroth moment of the full system.

𝐦0 = 𝐂𝐕(𝐖T𝐀𝐕)−𝟏

𝐖T𝐀𝐕𝐫0 = 𝐂𝐀−1𝐁 (74)

The first moment for the reduced system is given by Equation (75).

𝐦1 = 𝐂𝐕(𝐖𝐓𝐀𝐕)−𝟏

𝐖𝐓𝐄𝐕(𝐖𝐓𝐀𝐕)−𝟏

𝐖𝐓𝐁 (75)

Using (73), Equation (75) can be reduced to (76).

𝐦1 = 𝐂𝐕(𝐖𝐓𝐀𝐕)−𝟏

𝐖𝐓𝐄𝐀−1𝐛 (76)

If A-1EA-1B is in the column span of V, there exists a vector r1 as shown in Equation (77).

𝐕𝐫𝟏 = 𝐀−1𝐄𝐀−1𝐁 (77)

This relation can be used to write (76) as in Equation (78).

𝐦1 = 𝐂𝐕(𝐖𝐓𝐀𝐕)−𝟏

𝐖𝐓𝐀𝐕𝐫1 = 𝐂𝐀−1𝐄𝐀−1𝐁 (78)

Therefore, if A-1B and A-1EA-1B are in the column span of V, the zeroth and the first moments match.

This process can be continued to match the successive moments. To express this result in a simple

way, one may introduce the Krylov subspaces, defined in Equation (79).

𝒦(𝐆, 𝐯, 𝑞) = [𝐯 𝐆𝐯 𝐆𝟐𝐯 ⋯ 𝐆𝐪−𝟏𝐯 𝐆𝐪𝐯] (79)

Using this definition, q moments of the transfer function of the full system are matched if the matrix

V is given by Equation (80).

𝐕 = 𝒦(𝐀−𝟏𝐄, 𝐀−𝟏𝐁, 𝑞) (80)

It must be pointed out that nothing was imposed over the matrix W. A simple choice for this matrix is

to make it equal to V. However, one may use a very similar argument to show that W may be chosen

in order to match even more moments. A very similar argument shows that if W is chosen in

accordance to Equation (81), then other q moments of the full system are matched.

𝐖 = 𝒦(𝐀−𝐓𝐄𝐓, 𝐀−𝐓𝑪𝑻, 𝑞) (81)

In some special systems, e.g. impedance probing, both of the subspaces are the same. If A and E are

symmetric and CT = B, there is no need to calculate both subspaces.

3. Circuit Simulation

This section is largely related to the work in [4].

Before introducing the linear equations for circuit simulation, some notation and important

concepts are introduced. One way to model a circuit into the computer is by an abstraction

called graph. A graph, denoted by G(V, E), is a group of two sets: the set of vertices (V) and the

set of edges (E), containing tuples of elements of V.

For the case of circuit representation, an edge can be tough as a circuit component (resistor,

capacitor, current source, etc.) and the nodes are the connection points. In this particular

analysis, only resistors, capacitors, inductors and current sources will be considered. These

components and its orientations are illustrated in Figure 2.

Figure 2 - Circuit elements with voltage and current orientations

Suppose the existence of an oriented graph representing the circuit. In order to translate this

structure into matrix notation, one utilizes an edge-node incidence matrix (𝐀 ∈ ℝ|V|,|E|) with its

elements satisfying 𝐀ij ∈ {−1, 0, 1}. Each column of this matrix is directly associated with an

edge of the underlying graph and each row with a node. To build the matrix, each column must

contain exactly one entry 1, one entry -1 and |V| - 2 zero entries. The nonzero elements must be

placed accordingly with the incidence of the edges into the nodes. The graph orientation is used

to determine the sign of the entry.

From this matrix, it is possible to derive a reduced matrix excluding the line representing the

node of known potential, usually the ground node. This new matrix is denoted by �̃�. If the edges

of this matrix are placed in a way that components of the same kind are side by side, some

submatrices can be identified as shown in (82).

�̃� = [𝐀s 𝐀r 𝐀c 𝐀l] (82)

Using these submatrices is possible to write the Kirchhoff’s circuit laws for the circuit as shown

in Equation (83). The vectors 𝐢r, 𝐢c, 𝐢l e 𝐢s contains the currents of each of the different types of

circuit elements.

𝐀r𝐢r + 𝐀c𝐢c + 𝐀l𝐢l = 𝐀s𝐢s (83)

Direct application of resistors and capacitors terminal relations leads to Equation (84).

𝐀r𝐑−1𝐯𝑟 + 𝐀c𝐂d𝐯𝑐

dx+ 𝐀l𝐢l = 𝐀s𝐢s (84)

i

V

i

V

i

V

i

V

The relation between the voltage in each of the elements and the potential of each of the nodes

of the circuit is given by Equation (85). The index i is a placeholder and can be replaced by r, c, l

or s.

𝐀iT𝐯 = 𝐯i (85)

Using this relation, Equation (84) can be rewritten as in Equation (86).

𝐀r𝐑−1𝐀rT𝐯 + 𝐀c𝐂𝐀c

Td𝐯

dx+ 𝐀l𝐢l = 𝐀s𝐢s (86)

Writing the inductors terminal voltage and using Equation (85), it is possible to write Equation

(87). This allows obtaining a unified equation with the node voltages and currents in the

inductors as unknowns. This is presented in Equation (88).

𝐀iT𝐯 = 𝐯i (87)

[𝐀c𝐂𝐀𝑐T 𝟎

𝟎 −𝐋]

𝑑

𝑑𝑥[𝐯n

𝐢l] = − [

𝐀r𝐑−1𝐀rT 𝐀i

𝐀iT 𝟎

] [𝐯n

𝐢l] + [

𝐀𝑠

𝟎] 𝐢𝑠

(88)

If formulated in this fashion, both right and left Krylov subspaces are the equal. The same

phenomenon happens with the original and dual systems in the Proper Orthogonal Decomposition.

This allows calculating only one subspace and having the precision achieved by two subspaces in the

general case.

4. Application Example

The formulation developed in the last section is used to build an electric circuit as shown in Figure 3.

This circuit consists basically in a current source in parallel with a capacitor that feeds a ladder of

adjustable size that contains a pattern of inductance, resistance and capacitance. The output is an

ordinary resistor whose choice is independent of the other.

Figure 3 - RLC ladder

For this experiment a pattern as indicated in Table 4 was chosen. The Bode diagram for the

impedance measured at the terminals of the current source is plotted in Figure 4.

Table 4 - Value patterns for the RLC ladder

Parameter Value

Input capacitor 1 pF Output resistor 1 Ω Resistor pattern 1 Ω Inductor pattern 1 nH, 10 nH, 1μH Capacitor pattern 1pF, 1nF, 10nF

Figure 4 – Bode diagram of the system

The resulting reduced order models for an order equal to 20 is shown for the POD method in Figure

5. The sampling time was one nanosecond. Figure 6 shows the result for the Krylov method with 12

logarithmically distributed expansion points for the same system. Finally, Figure 7 shows the result

for a Balanced Truncation method.

Figure 5 – POD Approximation

Figure 6 – Krylov Approximation

Figure 7 – Balanced Truncation

5. Appendix – Mathematical development

Gradient property

𝑛 = ∑ 𝑢𝑘𝑣𝑘

𝑘

(89)

𝑑 𝑛

𝑑𝑥𝑖= ∑

𝑑𝑢𝑘

𝑑𝑥𝑖𝑣𝑘 + 𝑢𝑘

𝑑𝑣𝑘

𝑑𝑥𝑖𝑘

(90)

∇𝑥(𝑢𝑇𝑣) = (∇𝑥𝑢𝑇)𝑣 + (∇𝑥𝑣𝑇)𝑢 (91)

First gradient of the trace

𝑡𝑟(𝑉𝑇𝐾𝑉) = ∑ ∑ 𝑉𝑙𝑢 ∑ 𝐾𝑙𝑘𝑉𝑘𝑢

𝑘𝑙𝑢

(92)

𝑑 𝑡𝑟(𝑉𝑇𝐾𝑉)

𝑑𝑉𝑖𝑗= ∑ ∑ (

𝑑𝑉𝑙𝑢

𝑑𝑉𝑖𝑗∑ 𝐾𝑙𝑘𝑉𝑘𝑢

𝑘

+ 𝑉𝑙𝑢 ∑ 𝐾𝑙𝑘

𝑑𝑉𝑘𝑢

𝑑𝑉𝑖𝑗𝑘

)

𝑙𝑢

(93)

𝑑 𝑡𝑟(𝑉𝑇𝐾𝑉)

𝑑𝑉𝑖𝑗= ∑ 𝐾𝑖𝑘𝑉𝑘𝑗

𝑘

+ ∑(𝐾𝑇)𝑖𝑙𝑉𝑙𝑗

𝑙

(94)

∇𝑉 𝑡𝑟(𝑉𝑇𝐾𝑉) = 𝐾𝑉 + 𝐾𝑇𝑉 (95)

Second gradient of the trace

𝑡𝑟(𝐴𝑇𝑉𝑇𝑉) = ∑ ∑ 𝐴𝑙𝑢 ∑ 𝑉𝑘𝑙𝑉𝑘𝑢

𝑘𝑙𝑢

(96)

𝑑 𝑡𝑟(𝐴𝑇𝑉𝑇𝑉)

𝑑𝑉𝑖𝑗= ∑ ∑ 𝐴𝑙𝑢 ∑ (

𝑑𝑉𝑘𝑙

𝑑𝑉𝑖𝑗𝑉𝑘𝑢 + 𝑉𝑘𝑙

𝑑𝑉𝑘𝑢

𝑑𝑉𝑖𝑗)

𝑘𝑙𝑢

(97)

𝑑 𝑡𝑟(𝐴𝑇𝑉𝑇𝑉)

𝑑𝑉𝑖𝑗= ∑ 𝑉𝑖𝑢(𝐴𝑇)𝑢𝑗

𝑢

+ ∑ 𝑉𝑖𝑙𝐴𝑙𝑗

𝑙

(98)

∇𝑉 𝑡𝑟(𝐴𝑇𝑉𝑇𝑉) = 𝑉𝐴𝑇 + 𝑉𝐴 (99)