recursive bayesian estimation using gaussian sums

8/11/2019 Recursive Bayesian Estimation Using Gaussian Sums

1/15


2/15


3/15

R e c u r s i v e B a y e s i a n e s t i m a t i o n u s i n g g a u s s i a n s u m s 4 6 7

s u g g e s t e d b r i e f l y b y A O K t [ 1 1 ] . M o r e r e c e n t l y ,

CAMERON [12] and Lo [13] have as sum ed t ha t th e

a p r i o r i

d e n s i t y f u n c t i o n f o r t h e i n i t i a l s t a t e o f

l i n e a r s y s t e m s w i t h g a u s s i a n n o i s e s e q u e n c e s h a s

t h e f o r m o f a g a u s s ia n s u m .

2 .1 . T h e o r e t ic a l f o u n d a t i o n s o f t h e a p p r o x i m a t D n

C o n s i d e r a p r o b a b i l i t y d e n s i t y f u n c t i o n p w h i c h

i s a s s u m e d t o h a v e t h e f o l l o w i n g p r o p e r ti e s .

1 . p i s de f ined and cont inuous a t a l l bu t a f in i t e

n u m b e r o f l o ca t io n s

2. Ioop x ) d x = 1

3. p x)>_ 0 for a l l x

I t is c o n v e n i e n t a l t h o u g h n o t n e c e s s a r y t o c o n s i d e r

o n l y sc a l a r r a n d o m v a r i a b l es . T h e g e n e r a l i z a t i o n

t o t h e v e c t o r c a s e i s n o t d i f f i c u lt b u t c o m p l i c a t e s th e

p r e s e n t a t i o n a n d , i t i s f e l t , u n n e c e s s a r i l y d e t r a c t s

f r o m t h e b a s i c i d e a s.

T h e p r o b l e m o f a p p r o x i m a t i n g p c a n b e c o n -

v e n i e n t l y c o n s i d e r e d w i t h i n t h e c o n t e x t o f d e l t a

fami l i e s of pos i t ive type [14] . Bas ica l ly , t hese a re

f a m i l ie s o f f u n c t i o n s w h i c h c o n v e r g e t o a d e l t a , o r

i m p u l s e , f u n c t i o n a s a p a r a m e t e r c h a r a c t e r i z i n g t h e

f a m i l y c o n v e r g e s t o a l i m i t v a l u e . M o r e p r e c is e ly ,

l e t { 6 4} b e a f a m i l y o f f u n c t i o n s o n ( - o % o o) w h i c h

are in t egrab le over every in te rva l . Th i s is ca l l ed a

d e l t a f a m i l y o f p o s i ti v e t y p e i f t h e f o l l o w i n g c o n -

di t ions a re s a t i s f i ed .

(i)

a a

6 ~ x ) d x t e n d s t o o n e a s 2 t e n d s t o s o m e

l imi t va lue ; to for some a .

( i i ) F o r e v e r y c o n s t a n t y > _ 0 , n o m a t t e r h o w

s m a l l, 6 x t e n d s t o z e r o u n i f o r m l y f o r

7 < [ x [ < ~ a s 2 t e n ds t o 2 0.

(iii) 6 z x ) > 0 for a l l x and 2 .

U s i n g t h e d e l t a f a m i l ie s , th e f o l l o w i n g r e s u l t c a n

b e u s e d f o r t h e a p p r o x i m a t i o n o f a d e n s i ty f u n c t io n

p .

T h e o r e m 2.1 . Th e sequence p x x ) w h i c h i s f o r m e d

b y t h e c o n v o l u t i o n o f 6 z a n d p

p a x ) = f ~ oo c S a x - u ) p u ) d u

(7)

c o n v e rg e s u n i f o r m l y t o p x ) o n e v e r y i n t e r i o r s u b -

i n t e r v a l o f ( - ~ , o o) .

Fo r a p ro of of th i s re su l t , s ee KOREVAAR [14] .

W h e n p h a s a f i n i t e n u m b e r o f d i s c o n t i n u i t i e s , t h e

t h e o r e m i s s t i l l v a l i d e x c e p t a t t h e p o i n t s o f d i s -

c o n t i n u i t y . I t s h o u l d b e n o t e d t h a t e s s e n t ia l l y t h e

sam e resu l t i s g iven by T heo rem 2 .1 in FELLER[15].

I f { 6a } i s r e q u i r e d t o s a t i sf y t h e c o n d i t i o n t h a t

i t fo l l o w s f r o m e q u a t i o n ( 7 ) t h a t P a is a p r o b a b i l i t y

d e n s i t y f u n c t i o n f o r a l l 2.

I t i s b a s i c a ll y t h e p r e s e n c e o f t h e g a u s s i a n w e i g h t -

i n g f u n c t i o n t h a t h a s m a d e t h e E d g e w o r t h e x p a n -

s i o n a t t r a c t i v e f o r u s e i n t h e B a y e s i a n r e c u r s i o n

r e l a ti o n s . T h e o p e r a t i o n s d e f i n e d b y e q u a t i o n s

( 3 - 6 ) a r e s i m p l if i e d w h e n t h e

a p r i o r i

dens i t i e s a re

g a u s s i a n o r c l o s e ly r e l a t e d t o t h e g a u s s ia n . B e a r i n g

t h i s in m i n d , t h e f o l l o w i n g d e l ta f a m i l y i s a n a t u r a l

c h o i c e f o r d e n s i ty a p p r o x i m a t i o n s . L e t

,L x) A_U~ x)

= (2rc22) x p [ - x 2 / i2 ] .

( 8 )

I t i s s h o w n w i t h o u t d i f f ic u l ty t h a t N a x ) f o r m s a

d e l t a f a m i l y o f p o s it i v e ty p e a s ; t ~ 0 . T h a t i s , a s t h e

v a r i a n c e t e n d s t o z e r o , t h e g a u s s i a n d e n s i t y t e n d s

t o t h e d e l t a f u n c t i o n .

U s i n g e q u a t i o n s ( 7 ) a n d ( 8) , t h e d e n s i t y a p p r o x i -

m a t i o n p ~ is w r i t t e n a s

oo

P z x ) = p u ) N z x - u ) d u .

(9)

00

I t i s t h i s f o r m t h a t p r o v i d e s t h e b a s i s f o r t h e

g a u s s ia n s u m a p p r o x i m a t i o n t h a t i s t h e s u b j e ct o f

th i s d i scuss ion .

W h i l e e q u a t i o n ( 9 ) i s a n i n t e r e s ti n g r e s u lt , i t d o e s

n o t i m m e d i a t e l y p r o v i d e t h e a p p r o x i m a t i o n t h a t

c a n b e u s e d f o r s p e c if ic a p p l i c a t i o n . H o w e v e r , i t is

c l e a r t h a t p u ) N x x - u ) i s i n t egrable on ( - o% oo)

a n d i s a t l e a s t p i e c ew i s e c o n t i n u o u s . T h u s , ( 9) c a n

i t s e lf b e a p p r o x i m a t e d o n a n y f i n i t e i n t e r v a l b y a

R i e m a n n s u m . I n p a r t ic u l a r , co n s i d e r a n a p p r o x i -

m a t i o n o f P z o v e r s o m e b o u n d e d i n t er v a l ( a, b )

g i v e n b y

/1

p .. /x ) = ~ ~__Ep(x~)N~(x- x,)[ ~,- 4 ,- i ] (i0)

w h e r e t h e i n t e r v a l ( a , b ) i s d i v i d e d i n t o n s u b -

in te rva l s by se lec t ing poin t s ~ , such th a t

a = ~ o < ~ l < . . . ~ , = b .

I n e a c h s u b i n t e r v a l, c h o o s e t h e p o i n t x ~ s u c h t h a t

f

~

P X , ) [ ~ i - 4 , - 1 ] = p x ) d x

~ I

w h i c h i s p o s s i b le b y t h e m e a n - v a l u e th e o r e m . T h e

c o n s t a n t k i s a n o r m a l i z i n g c o n s t a n t e q u a l t o

f

~ o 6 z ( x ) d x = l

k > x , d x


4/15

4 6 8 H .W . SOREN SO N an d D . L . A LSPA CH

a n d i n s u r e s t h a t p . , x i s a d e n s i t y f u n c t i o n . C l e a r l y ,

f o r ( b - a ) s u f fi c i e n tl y l a r g e , k c a n b e m a d e a r b i -

t r a r il y d o s e t o 1. N o t e t h a t i t f o l lo w s t h a t

T h u s , o n e c a n a t t e m p t t o c h o o s e

~ i , # i a i ( i= 1 , 2 . . . . , n )

n

k , '= p ( x i ) [ { , - 4 , - a] = 1

(1 1 )

s o t h a t p . , a e s s e n t i a l ly i s a c o n v e x c o m b i n a t i o n o f

g a u s s i a n d e n s i t y f t m c t i o n s N a . I t i s b a s i c a l l y t h i s

f o r m t h a t w i l l b e u s e d i n a l l f u t u r e d i s c u s s i o n a n d

w h i c h i s r e f e r r e d t o h e r e a f t e r a s t h e gauss ian sum

approximation.

I t i s i m p o r t a n t t o r e c o g n i z e t h e

p . , a t h a t a r e f o r m e d i n t hi s m a n n e r a r e v a li d p r o b a -

b i l i t y d e n s i t y f u n c t i o n s f o r a l l n , 2 .

2 .2 . Implem enta t ion o f the gaussian sum approxima-

tion

T h e p r e c e d i n g d i s c u s s i o n h a s i n d i c a t e d t h a t a

p r o b a b i l i t y d e n s i t y f u n c t i o n p t h a t h a s a f in i t e

n u m b e r o f d i s c o n ti n u i ti e s c a n b e a p p r o x i m a t e d

a r b i t r a r i l y c l o s e l y o u t s i d e o f a r e g i o n o f a r b i t r a r i l y

s m a l l m e a s u r e a r o u n d e a c h p o i n t o f d i s c o n t in u i t y

b y t h e g a u s s i a n s u m p , . a a s d e f i n e d i n e q u a t i o n ( 1 0 ) .

T h e s e a s y m p t o t i c p r o p e r t i e s a r e c e r t a i n l y n e c e s s a ry

f o r t h e c o n t i n u e d i n v e s t i g a t i o n o f t h e g a u s s i a n s u m .

H o w e v e r , f o r p r a c t i c a l p u r p o s e s , i t i s d e s i r a b l e , i n

f a c t i m p e r a ti v e , t h a t p c a n b e a p p r o x i m a t e d t o

w i t h i n a n a c c e p t a b l e a c c u r a c y b y a r e l a t i v e l y s m a l l

n u m b e r o f t e r m s o f th e s e ri es . T h i s r e q u i r e m e n t

f u r n i s h e s a n a d d i t i o n a l f a c e t t o t h e p r o b l e m t h a t i s

c o n s i d e r e d i n t h i s s e c t i o n .

F o r t h e s u b s e q u e n t d i s c u s s i o n , i t i s c o n v e n i e n t t o

w r i t e t h e g a u s si a n s u m a p p r o x i m a t i o n a s

w h e r e

p . ( x ) = ~ ~ l N . , ( x - p t ) ( 12 )

i =1

~ = 1 ; ~ > 0 f o r a l l i .

i =1

T h e r e l a t i o n o f e q u a t i o n ( 1 2) t o e q u a t i o n ( 1 0) i s

o b v i o u s b y i n sp e c t i on . U n l i k e e q u a t i o n ( 1 0 ) i n

w h i c h t h e v a r i a n c e 2 2 i s c o m m o n t o e v e r y t e r m o f

t h e g a u s s i a n s u m , i t h a s b e e n a s s u m e d t h a t t h e

v a r i a n c e a i 2 c a n v a r y f r o m o n e t e r m t o a n o t h e r .

T h i s h a s b e e n d o n e t o o b t a i n g r e a t e r f l e x i bi l i ty f o r

a p p r o x i m a t i o n s u s i n g a f i n i te n u m b e r o f t er m s .

C e r t a i n l y , a s t h e n u m b e r o f t e r m s i n c r e a s e , i t i s

n e c e s s a r y t o r e q u i r e t h a t t h e a i t e n d t o b e c o m e

e q u a l a n d v a n i s h .

T h e p r o b l e m o f c h o o s i n g t h e p a r a m e t e r s ~ , / ~ i , a~

t o o b t a i n th e " b e s t " a p p r o x i m a t i o n p , t o s o m e

d e n s i t y f u n c t i o n p c a n b e c o n s i d e r e d . T o d e f i n e t h i s

m o r e p r e c i s e l y , c o n s i d e r t h e L k n o r m . T h e d i s ta n c e

b e t w e e n p a n d p , c a n b e d e f i n e d a s

oO

p _ p , k = p ( x ) -

s o t h a t t h e d i st a n c e I I P - P ,[ [ i s m i n i m i z e d . A s t h e

n u m b e r o f t e r m s n i n c re a s e s a n d a s t h e v a r i a n c e

d e c r e a s e s t o z e r o , t h e d i s t a n c e m u s t v a n i s h . H o w -

e v e r , f o r f in i t e n a n d n o n z e r o v a r i a n c e , i t i s re a s o n -

a b l e t o a t t e m p t t o m i n i m i z e t h e d is t a n ce i n a m a n n e r

s u c h a s th i s . I n d o i n g t h i s , t h e s t o c h a s t i c e s t i m a t i o n

p r o b l e m h a s b e e n r e c a s t a t t h i s p o i n t a s a d e t e r -

m i n i s t i c c u r v e f i t t i n g p r o b l e m .

T h e r e a r e o t h e r n o r m s a n d p r o b l e m f o r m u l a t i o n s

t h a t c o u l d b e c on s i d e re d . I n m a n y p r o b l e m s , i t

m a y b e d e s i r a b l e t o c a u s e t h e a p p r o x i m a t i o n t o

m a t c h s o m e o f t h e m o m e n t s , f o r e x a m p l e , t h e m e a n

a n d v a r i a n c e , o f t h e t r u e d e n s i t y e x a c t ly . I f t h i s

w e r e a r e q u i r e m e n t , t h e n o n e c o u l d c o n s i d e r t h e

m o m e n t s a s c o n s t r a i n t s o n t h e m i n i m i z a t i o n

p r o b l e m a n d p r o c e e d a p p r o p ri a t e l y . F o r e x a m p le ,

i f t h e m e a n a s s o c i a t e d w i t h p i s p , t h e n t h e c o n -

s t r a in t t h a t p , h a v e m e a n v a l u e p w o u l d b e

U ~ n ~ i al -- i

= ~ ai l t l. (1 4 )

i =1

T h u s , e q u a t i o n ( 14 ) w o u l d b e c o n s i d e r e d i n a d d i t io n

t o t h e c o n s t r a i n t s o n t h e o q s t a t e d a f t e r e q u a t i o n

(12).

S t u d ie s r e l a te d t o t h e p r o b l e m o f a p p r o x i m a t i n g

p w i t h a s m a l l n u m b e r o f te r m s h a v e b e e n c o n d u c t e d

f o r a la r g e n u m b e r o f d e n s i t y f u n c t i o n s . T h e s e i n -

v e s t i g a t i o n s h a v e i n d i c a t e d , n o t s u r p r i s i n g l y , t h a t

d e n s i t i e s w h i c h h a v e d i s c o n t i n u i t i e s g e n e r a l l y a r e

m o r e d i ff ic u l t t o a p p r o x i m a t e t h a n a r e c o n t i n u o u s

f u n c t i o n s . T h e r e s u l t s f o r t w o d e n s i t y f u n c t i o n s , t h e

u n i f o r m a n d t h e g a m m a , a r e d is c u s se d b e l o w . T h e

u n i f o r m d e n s i t y is d i s c o n t i n u o u s a n d i s o f i n t e r e s t

f r o m t h a t p o i n t o f v ie w . T h e g a m m a f u n c t i o n i s

n o n z e r o o n l y f o r p o s i t i v e v a l u e s o f x s o i s a n

e x a m p l e o f a n o n s y m m e t r i c d e n s i t y t h a t e x t e n d s

o v er a semi- in f in i te r an g e .

C o n s i d e r t h e f o l l o w i n g u n i f o r m d e n s it y f u n c t i o n

J' f o r - 2 < x _ 2

p ( x ) = ~ 0 e l s e w h e r e

1 5 )

T h i s d i s t ri b u t i o n h a s a m e a n v a l u e o f z e r o a n d

v a r i a n c e o f 1 .3 3 3 .

T w o d i f fe r e n t m e t h o d s o f f i tt i ng e q u a t i o n ( 1 5)

h a v e b e e n c o n s i d e r e d . F i r s t , c o n s i d e r a n a p p r o x i -

m a t i o n t h a t i s s u g g e s te d d i r e c t l y b y ( 1 0 ) a n d r e f e r r e d

t o s u b s e q u e n t l y a s a Theorem Fi t . T h e p a r a m e t e rs

o f t h e a p p r o x i m a t i o n a r e c h o s e n i n t h e f o l lo w i n g

g e n e r a l m a n n e r .


5/15


( 1) S e l e c t t h e m e a n v a l u e p~ o f e a c h g a u s s i a n

s o t h a t t h e d e n s i t i e s a r e e q u a l l y s p a c e d o n

( - 2 , 2 ) . B y a n a p p r o p r i a t e lo c a t i o n o f t h e

d e n s i t i e s t h e m e a n v a l u e c o n s t r a i n t ( 1 4 ) c a n

b e s a t is f ie d i m m e d i a t e l y .

(2 ) The we ight ing fac tors 0q a re s e t equa l to

i / n s o

~ t = l

t = l

( 3) T h e v a r i a n c e 12 o f e a c h g a u s s i a n i s th e s a m e

a n d i s s el e c te d s o t h a t t h e L 1 d i s t a n c e b e t w e e n

p a n d p , i s m i n i m i z e d .

T h i s a p p r o x i m a t i o n p r o c e d u r e r e q u i re s o n l y a o n e -

d i m e n s i o n a l s e a r c h t o d e t e r m i n e 2 .

T o i n v e s t i g a t e t h e a c c u r a c y a n d t h e c o n v e r g e n c e

o f t h e a p p r o x i m a t i o n , t h e n u m b e r o f t e r m s i n th e

s u m w a s v a r i e d . F i g u r e l ( a ) s h o w s t h e a p p r o x i m a -

t i o n w h e n 6 , 1 0 , 2 0 a n d 4 9 t e r m s w e r e i n c l u d e d . I t

i s i n t e r e s t i n g t o o b s e r v e t h a t t h e a p p r o x i m a t i o n

r e t ai n s t h e g e n e r a l c h a r a c te r o f t h e u n i f o r m d e n s i t y

e v e n f o r t h e s i x -t e r m ca s e . A s s h o u l d b e e x p e c t e d ,

t h e l a r g e s t e r r o r s a p p e a r i n t h e v i c i n i t y o f t h e d i s -

c o n t i n u i t i e s a t + 2 . T h e s e a p p r o x i m a t i o n s e x h i b i t

a n a p p a r e n t o s c i l l a t i o n a b o u t t h e t r u e v a l u e t h a t i s

n o t v i s u a l l y s a t is f y in g . T h i s o s c i l l a t io n c a n b e

e l i m i n a t e d b y u s i n g a s l i g h t l y l a r g e r v a l u e f o r t h e

v a r i a n c e o f t h e g a u s s i a n t e r m s a s i s d e p i c t e d i n

F ig . l (b ) .

T h e s e c o n d a n d f o u r t h m o m e n t s a n d t h e L 1 e r r o r

a r e l i s te d i n T a b l e 1 f o r t h e s e t w o s e ts o f a p p r o x i m a -

t i o n . T h e i n d i v i d u a l t e r m s w e r e l o c a t e d s y m -

m e t r i ca l ly a b o u t z e r o s o t h a t t h e m e a n v a l u e o f t h e

g a u s s i an s u m a g r ee s w i t h t h a t o f t h e u n i f o r m d e n -

s i ty i n a ll c as e s. N o t e t h a t f o r t h e b e s t f it t h e e r r o r

i n t h e v a r i a n c e i s o n l y 1 . 25 p e r c e n t w h e n 2 0 t e r m s

a r e u s e d . A s s h o u l d b e e x p e c t ed , h i g h e r o r d e r

[a.

t~

13.

O . S -

o )

0 . 4

0 . 3

0 . 2

o I /

3 2 . 1

0 . 5 -

(c)

0 . 4

0 . 3

0.2

'-,.b ' 6

X

0 . 5 -

b )

0 . 4

0 . 3

0 . 2 -

' I -~ 0 2 0 3 0 - 3 - 0 - 2 - 0

i , , i i f i t i ,

- I ' 0 0 I ' 0 2 ' 0 3 ' 0

0.1 { '

r , i i , i , 1 , i

- 3 ' 0 - 2 0 - I 0 0 | ' 0

X

Fzo . 1 . Gau ss ian

[ ~ s e o r c h f i t

. . . . . I 0 t e r m b e s t t h e o r e m f i t

. .. .. .. .. I 0 t e r m s m o o t h e d t h e o r e m f i t

k

\

2 . 0 3 . 0

s u m a p p r o x i m a t i o n s o f u n i f or m

d en s i ty fu n c t io n .

( a ) B e s t t h e o r e m f i t - - 6 , 1 0, 2 0 a n d 4 9 t e r m a p p r o x i -

m a t i o n s .

( b ) S m o o t h e d t h e o r e m f i t - - 6 , 1 0 , 2 0 a n d 4 9 t e r m

a p p r o x i m a t i o n s .

(C) L 2 Search fit com parison,


6/15

4 7 0 H . W . S OR EN SO N a n d D . L . A L SP A CH

m o m e n t s c o n v e r g e m o r e s l ow l y si n ce e r ro r s f a r t h e s t

a w a y f r o m th e m e a n a s su m e m o r e i m p o r t an c e . F o r

e x a m p l e , t h e f o u r t h c e n t r a l m o m e n t h a s a n e r r o r o f

5 p e r c en t f o r t h e 2 0 -t e r m a p p r o x i m a t i o n . T h e

e r r o r s i n t h e m o m e n t s o f t h e s m o o t h e d f it a r e

a g g r a v a t e d o n l y s l i gh t l y a l t h o u g h t h e L I e r r o r

i n c r e a s e s i n a n o n t r i v i a l m a n n e r .

TABLE 1. UNIFORM DENSITY APPROXIMATION

Fourth

central L 1

Variance mome nt error

rue values 1 333 3.200 - -

o f t h e n u m b e r o f t e r m s i n v o l v e d , o b t a i n i n g a s e ar c h

f i t i s s i g n i f i c a n t l y m o r e d i f f i c u l t t h a n o b t a i n i n g a

t h e o r e m f it f o r th e s a m e n u m b e r o f t e rm s . T h u s , t h e

t h e o r e m f it m a y b e m o r e d e s i r a b l e f r o m a p r a c t i c a l

s t a n d p o in t . T h e m o m e n t s a n d L 1 e r r o r f o r t h e

s e a r c h f i t i s a l s o i n c l u d e d i n T a b l e 1 . T h e s e v a l u e s

a r e s l i g h tl y b e t t e r t h a n t h e t h e o r e m f it in v o l v i n g t e n

t e r m s . I t i s i n t e r e s t i n g i n F i g . l ( c ) t o n o t e t h e

" s p i k e s " t h a t h a v e a p p e a r e d a t t h e p o i n t s o f d is -

c o n t i n u i t ie s . T h i s a p p e a r s t o b e a n a l o g o u s t o th e

G i b b s p h e n o m e n o n o f F o u r i e r s er ie s.

T h e s e c o n d e x a m p l e t h a t i s d i s c u s s e d h e r e i s t h e

g a m m a d e n s i t y f u n c t i o n . I t is d e f i n e d a s

B e s t t h e o re m f i t

6 term s 1.581 5-320 0.2199

10 terms 1.417 3.884 0.1271

20 term s 1.354 3.363 0.0623

49 terms 1.336 3.226 0.0272

f

0 f o r x < 0

p ( x ) =

x 3 e X f o r x > 0 .

6

( 1 6 )

S m o o t h e d t h e o re m f i t

6 term s 1 "690 6.387 0.2444

10 terms 1.456 4.224 0.1426

20 terms 1.363 3"442 0.0701

49 terms 1.338 3.238 0.0280

L 2 search fi t

6 terms 1.419 3.626 0.0968

A s a n a l t e r n a t i v e a p p r o a c h , t h e p a r a m e t e r s ~ i

# t, tr2 w e r e c h o s e n t o m i n i m i z e t h e L 2 d i s ta n c e

w h i c h i s h e r e a f t e r r e f e r r e d t o a s a n L 2 s e a r c h f i t.

T h e s e r e s u l ts a r e s u m m a r i z e d i n F i g . 1 c ) f o r n = 6 .

I n c l u d e d f o r c o m p a r i s o n i n t h i s fi g u r e a r e t h e 1 0 -

t e r m t h e o r e m f it s f r o m F i g s. l ( a ) a n d l ( b ) . B e c a u s e

T h e d i s t r i b u t i o n h a s a m e a n v a l u e o f 4 a n d s e c o n d ,

t h i rd , a n d f o u r t h c e n t r al m o m e n t s o f 4 , 8 a n d 7 2

r e s p e c t i v e l y .

F i r s t , c o n s i d e r a t h e o r e m f it o f t h i s d e n s i t y in

w h i c h t h e m e a n v a l ue s a r e d i st r i b u te d u n i f o r m l y o n

( 0 , 10 ). F o r t h e u n i f o r m d e n s i t y , t h e u n i f o r m p l a c e -

m e n t w a s n a t u r a l ; f o r t he g a m m a d e n s i t y i t i s n o t

a s a p p r o p r i a t e . F o r e x a m p l e , f o r n = 6 o r 1 0 , i t i s

s e e n i n F i g . 2 (a ) t h a t t h e a p p r o x i m a t i n g d e n s i t y i s

n o t a s g o o d , a t l e a s t v i s u a l l y , a s o n e m i g h t h o p e .

T h e f i r s t f o u r c e n t r a l m o m e n t s a r e l is t ed i n T a b l e 2 .

C l e a r l y , t h e h i g h e r o r d e r m o m e n t s c o n t a i n l a r g e

e r r o r s a n d e v e n t h e m e a n v a l u e i s i n c o r r e c t i n

c o n t r a s t w i t h t h e u n i f o r m d e n s i t y .

TABLE 2. GAMM A DENSITY

APPROXIMATION

T h i rd F o u r th

c e n t r a l

central L1

Mean Variance mom ent mom ent Error

True values

4 4 8 72

T h e o r em f i t

6 terms in (0, 10) 3"94 4"345 5"496 60.78 0"119

10 terms in (0, 10) 3.94 3.861 4-941 47.84 0.053

20 terms in (0, 10) 3.93 3.611 4-653 41.23 0.023

20 terms in (0, 12) 4.00 4.206 7-477 71.41 0.042

L2 search f i t

1 terms in (0, 10) 3'51 3"510 0 10.53 0"203

2 terms in (0, 10) 3"82 3"427 3.04 34"96 0"078

3 terms in (0, 10) 3.91 3.632 4"711 43-39 0"036

4 term s in (0, 10) 3"95 3-744 5"682 49-57 0"018


7/15

R e c u r s i v e B a y e s i a , e s t i m a t i o n u s i n g g a u s s ia n s u m s 4 7 1

Lt.

(L

O - Z

0 ' 2

0 .1

0 .1

0 , 0

0 . 0

- 2 . 0

/J

0 0

a)

1 0 - t e r m

. . . . . . . . . . . . 0 . Z

6 - t e r m . . . . . . . .

o . 2

0 . 1

LL

Q

0 -

0 1

;X

~ *k O 0

2 0

4 0 6 0 8 ' 0

I 0 0 1 2 ' 0

x

: - ' 0 .0 ' - : ~

1 4 " 0 - 2 0

0 0

( b )

i n t e r v a l s

0 ,10 ) . . . . . . .

( 0 , 1 2 ) . . . . . . . . . .

2 . 0 4 . 0 6 . 0 8 . 0 I 0 . 0 1 2 - 0 1 4 . 0

X

0 . 2

0 ' 2

o.,

rt

0 ' 1

I 0 ' 0 t

0 0 t x

- 2 - 0 0 . 0 2 - 0 4 . 0 6 . 0

X

F I O 2 G a u s s i a n s u m a p p r o x i m a t io n s o f g a m m a d e n s i ty

function.

(a) Best theorem fit- -6 an d 10 term approximations.

(b) Tw enty term approximations over different inter-

vals.

(c) L2 search fit: 3 and 4 term approximations.

c )

3 - t e r m . . . . . .

4 - t e r m . . . . . . . . . . .

8 ' 0 1 0 ' 0 1 2 " 0 1 4 " 0

T w o d i f f e r e n t 2 0 - t e r m a p p r o x i m a t i o n s a r e d e -

p i c t e d i n F i g. 2 ( b ) . I n o n e t h e m e a n v a l u e s o f t h e

g a u s s i a n t e r m s a r e s e l e c t e d i n t h e i n t e r v a l ( 0 , 1 0 ) ,

w h e r e a s t h e s e c o n d a p p r o x i m a t i o n i s d i st r ib u t e d i n

( 0 , 1 2 ) . N o t e i n t h e fi r st c a s e t h e g a u s s i a n s u m t e n d s

t o z e r o m u c h m o r e r a p i d ly t h a n t h e g a m m a f u n c t i o n

f o r x > 1 0. T h u s , t o i m p r o v e t h e a p p r o x i m a t i o n a n d

t h e m o m e n t s i t i s n e c e s s a r y t o i n c r e a s e t h e i n t e r v a l

o v e r w h i c h t e r m s a r e p l a c e d a n d t h e s e c o n d c u r v e

i n d i c a t e s t h e i n fl u e n c e o f t h i s c h a n g e . T h e r e s u l t s

o f t h e s e t w o c a s e s a r e i n c l u d e d i n F i g . 2 a n d i n d i c a t e

t h a t i n c r e a s i n g t h e i n t e r v a l o v e r w h i c h t h e a p p r o x i -

m a t i o n i s v a l i d h a s s i g n i f i c a n t l y i m p r o v e d t h e

m o m e n t s .

T h e t h e o r e m f i t p r o v i d e s a v e r y s im p l e m e t h o d

f o r ob t a i n i n g a n a p p r o x i m a t i o n . H o w e v e r , i t i s

c l e a r th a t b e t t e r r e s u l t s c o u l d b e o b t a i n e d b y c h o o s -

i n g a t le a s t t h e m e a n v a l u e s o f t h e i n d i v i d u a l

g a u s s ia n t e r m s m o r e c a r e fu l ly . C o n s i d e r n o w s o m e

L 2 se a r c h f i t s . In F ig . 2 ( c ) , th e s e a r c h f i t s f o r th r e e


8/15

472 H.W. SORENSON and D. L. ALSPACH

and four terms are depicted and the moments are

listed in Table 2. Clearly, values of the mean and

variance appear to be converging to the true values

of 4. Note also that the 4-term approximation is

considerably better than the 10-term theorem fit.

Thus, the search technique, while more difficult to

obtain, points out the desirability of judicious

placement of the gauss,an terms in order to obtain

the most suitable approximation.

3. LINEAR SYSTEMS WITH NONGAUSSIAN

NOISE

It is envisioned that the gauss,an sum approxima-

tion will be very useful in dealing with non-linear

stochastic systems. However, many of the

properties and concomitant difficulties of the

approximation are exhibited by considering linear

systems which are influenced by nongaussian noise.

As is well-known, the a p o s t e r i o r , density p ( X k / Z k )

is gauss,an for all k when the system is linear and

the initial state and plant and measurement noise

sequences are gauss,an. The mean and variance of

the conditional density are described by the Kalman

filter equations. When nongaussian distributions

are ascribed to the initial state and/or noise se-

quences,

p ( X k / Z k )

is no longer gauss,an and it is

generally impossible to determine p ( X k / Z k ) in a

closed form. Furthermore, in the linear, gauss,an

problem, the conditional mean, i.e. the minimum

variance estimate, is a linear function of the

measurement data and the conditional variance is

independent of the measurement data. These

characteristics are generally not true for a system

which is either non-linear or nongaussian.

For the following discussion, consider a scalar

system whose state evolves according to

X k = ~ k , k - l X k - 1 + W k - 1

(17)

and whose behavior is observed through measure-

ment data z k described by

z k = H k x ~ + V K. (18)

Suppose that the density function describing the

initial state has the form

lo

p(xo)= ~ ~ , o N t r ' i ( X o - , ; o ) . (19)

i = l

Assume that the plant and measurement noise

sequences (i.e.

{ W k }

and {Vk} are mutually inde-

pendent, white noise sequences with density func-

tions represented by

~k

p(w~)= Y t~,~Nqk,(w~--o),~) (20)

i = 1

m

P @ k ) : E ] ' i k N r , k ( O k - - V i k ) ( 2 1 )

i = 1

There are a variety of ways in which the gauss,an

sum approximation could be introduced. For

example, it is natural to proceed in the manner tha t

is to be discussed here in which the a p r i o r i distribu-

tions are represented by gauss,an sums as in equa-

tions (19), (20) and (21). This approach has the

advantage that the approximation can be deter-

mined off-line and then used directly in the Bayesian

recurs,on relations. An alternative approach

would be to perform the approximation in more of

an on-line procedure. Instead of approximating the

a p r i o r i

densities, one could deal with

p ( X J Z k )

in

equation (3) and the integrand in (4) and derive

approximations at each stage. This would be more

direct but has the disadvantage that considerable

computation may be required during the processing

of data. Discussion of the implementat ion of this

approach will not be attempted in this paper.

3 . 1. D e t e r m i n a t i o n o f t h e a posterior, d i s t r i b u t i o n s

Suppose that the

a p r i o r i

density functions are

given by equations (19, 20 and 21). In using these

representations in the Bayesian recurs,on relations,

it is useful to note the following properties of

gauss,an density functions

S e h o l i u m 3.1: For a4:0,

N . ( x - a y ) = 1 N . l . ( y - x / a ) .

(22)

a

S c h o l i u m 3.2:

where

N ~ , ( x - , i ) N , j ( x - , j )

2

- - 2 2 1 ~

- N t . , , + ~ j ( . , - . s ) N ~ , s ( x - . , j )

. , 4 . J 4

2 2

_ 2 _ _ i f , 7 j

O s

a ? + a ~ .

(23)

The proofs of (22) and (23) are omitted.

Armed with these two results it is a simple matter

to prove that the following descriptions of the

filtering and prediction densities are true.

T h e o r e m

3.1. Suppose that

P ( X k / Z k - O

is

described by

It ,

p ( X k / Z k - 1 ) = E a * k N , ; k ( X k -- ; k ) (24)

i = 1

Then,

p ( X k / Z k )

is given by

l k m k

p(~dz~)= E Z , j N . , ~ x ~ - ~ , j ) (25)

i = 1 j = I


9/15

R e c u r s i v e B a y e s i a n e s t i m a t i o n u s i n g g a u s s i a n s u m s 4 73

w h e r e

e~ , , .~ N ~(z~ - P~ ,. )

C i j = O t l k T j k N x ( z k P i j m ~ = 1

y ' 2 _ _ 1 t . /' 2 _ . '2 - - _ 2

- - ~ k O t k 1- r j k

, a l ~ H ~ r z

eq = I~ik4 ~ ,2 ~-- 7-Y 7, 2 L k - - V~k-- Hk tk]

~ t k - r l k t r j k

2 , '2 , '4 2 , '2 2 2

t ru = ~ i~ - t rt k H k / (a i~ H ~ + r ~ ) .

I t i s o b v i o u s t h a t t h e c u > 0 a n d t h a t

l k rak

2 Z c,~=~.

T h u s , e q u a t i o n ( 2 5 ) i s a g a u s s i a n s u m a n d f o r c o n -

v e n i e n e e o n e c a n r e w r i t e i t a s

I lk

p(x k /Z k ) = ~ . a t kN ~ , ~ (X k- - I~i k) (26)

t = 1

w h e r e n k = ( l k ) ( m k ) a n d t h e a ik , P i k a n d / q k a r e

f o r m e d i n a n o b v i o u s f a s h i o n f r o m t h e c u , tr u a n d

e U

T h e p r o o f o f t h e t h e o r e m i s s t r a i g h t f o r w a r d .

F r o m t h e d e f i n i ti o n o f th e m e a s u r e m e n t r e l a t i o n

( 18 ) a n d t h e m e a s u r e m e n t n o i s e d e n s i t y ( 21 ), o n e

sees tha t

m k

P(Zk/Xk) = ~ . , ~ k N , , k ( Z k - - H k x k - - V tk ) .

U s i n g t h i s a n d ( 2 4) i n (3 ) a n d a p p l y i n g t h e t w o

s c h o l i u m s , o n e o b t a i n s ( 25 ).

T h e p r e d i c t i o n d e n s i t y i s d e t e r m i n e d f r o m ( 4) a n d

l e a d s t o t h e f o l l o w i n g r e su l t .

T h e o r e m

3 .2 . A s s u m e t h a t

p ( x k / Z k )

i s g iven by

( 26 ). T h e n f o r t h e l i n e a r s y s te m (1 7 ) a n d p l a n t

n o i s e ( 20 ) t h e p r e d i c t i o n d e n s i t y

p ( X k + 1 / Z k )

is

n k $ k

p(x~+l/z~)= Y, 2 a ~j J /x~ + l

i = j= l

- Ck + 1, kl~ik - 09jk) (27)

w h e r e

2 2 2 2

2 u = ~ + l , k P lk + q yk

I t i s c o n v e n i e n t t o r e d e f i n e te r m s s o t h a t ( 27 ) c a n

b e w r i t t e n a s

l k + l

p x ~ +

~ l Z ~ ) = ~ ~ ( ~ + ~ ) N ~ ( ~ + , ) ( x k + ~ - ~ i ( ~ + ~)).

2 8 )

C l e a r l y , t h e d e f i n i t i o n o f p ( X o ) h a s t h e f o r m ( 2 8 )

a s d o e s t h e p ( x k / Z k - 1 ) a s s u m e d i n T h e o r e m 3 . 1 .

T h u s , i t f o l l o w s t h a t t h e g a u s s i a n s u m s r e p e a t

t h e m s e l v e s f r o m o n e s t a g e t o t h e n e x t a n d t h a t ( 2 6 )

a n d ( 2 8 ) c a n b e r e g a r d e d a s t h e g e n e r a l f o r m s t o r

a n a r b i t r a r y s t ag e . T h u s , t h e g a u s s i a n s u m c a n

a l m o s t b e r e g a r d e d a s a r e p r o d u c i n g d e n s i t y [ 1 6 ] .

I t i s i m p o r t a n t , h o w e v e r , t o n o t e t h a t t h e n u m b e r

o f t e r m s i n t h e g a u s s i a n s u m i n c r e as e s a t e a c h s t a g e

s o t h a t t h e d e n s i t y is n o t d e s c r i b e d b y a f i x e d

n u m b e r o f p a r am e t e r s . T h e d e n s i t y w o u l d b e t r u ly

r e p r o d u c i n g i f o n l y t h e i n i t i a l s t a t e w e r e n o n -

gauss ian , for exam ple , s ee Ref . [12]. I t i s c l ea r in

t h i s c a s e t h a t t h e n u m b e r o f te r m s i n t h e g a u s s i a n

s u m r e m a i n s e q u a l t o t h e n u m b e r u s e d t o d e f i n e

p ( x o ) so t h a t p ( X k / Z k ) i s d e s c r i b e d b y a f i x e d n u m b e r

o f p a r a m e t e r s .

T h e r e a r e s e v e r a l a s p e c t s t h a t r e q u i r e c o m m e n t

a t t h i s p o i n t . F i r s t , i f t h e g a u s s i a n s u m s f o r t h e a

p r i o r i

d e n s i t y a ll c o n t a i n o n l y o n e t e r m , t h a t i s , t h e y

a r e g a u s s i a n , t h e K a l m a n f i l t e r e q u a t i o n s a n d t h e

g a u s s i a n

a p o s t e r i o r i

d e n s i t y a r e o b t a i n e d . I n f a c t

t h e e u a n d a . . 2 i n (2 5) a n d t h e m e a n s a n d v a r i a n c e s

~J

21k2 in ( 2 7) e a c h r e p r e s e n t t h e K a l m a n f i l t e r e q u a -

t i o n s f o r t h e i j t h d e n s i t y c o m b i n a t i o n . T h u s , t h e

g a u s s i a n s u m i n a m a n n e r o 1 s p e a k i n g d e s c r i b e s a

c o m b i n a t i o n o f K a l m a n f i lt er s o p e r a ti n g i n c o n c e r t .

T o e x a m i n e t h i s t u r t h e r , c o n s i d e r t h e f i r s t a n d

s e c o n d m o m e n t s a s s o c i a t ed w i t h t h e p r e d i c t i o n a n d

f i l t e r ing d ens i t i e s .

T h e o r e m 3.3

l k m k

1 = 1 j = l

e E ( x ~ - ~ / ~ ) ~ l z d A _ p ~ / ~

l k m k

Y __ ., Y . , c i j [ o , j 2 + ~ k / , , - ~ t j) 2 ] 3 0 )

i = 1 j = l

E [ x k + ~ l Z d A e ~ + x / ~ = O k + ~ , k e ~ /k + E [ w d (31)

E [ x ~ + 1 - ~ + ~ ~ ) 2[

Z d = p k

+ ~ / k

2

= ~ k+ l , k2pk / k2 + E { [w k- -E (o gk ) ] 2} (32)

w h e r e

~k

t-----1

~k

E { E W k - - ~ ( ( D k ) ] 2 } = ~ ~ i k q i k 2 d l- ( .Oik 2) - - E 2 W k ) .

i - - - I

I n e q u a t i o n ( 2 9 ) , t h e m e a n v a l u e ~ k / k i s f o r m e d

a s t h e c o n v e x c o m b i n a t i o n o f th e m e a n v a l u e s 8 q

o f t h e i n d i v i d u a l t e r m s , o r K a l m a n f i l t e r s , o f t h e

g a u s s i a n su m . I t is

i m p o r t a n t t o r e c o g n i z e

t h a t t h e

c q , a s i s a p p a r e n t f r o m e q u a t i o n ( 25 ), d e p e n d u p o n

t h e m e a s u r e m e n t d a t a . T h u s , t h e c o n d i t i o n a l m e a n

is a n o n - l i n e a r f u n c t i o n o f t h e c u r r e n t m e a s u r e m e n t

d a t a .


10/15

4 7 4 H .W . SOREN SO N an d D . L . A LSPA CH

T h e c o n d i t i o n a l variance p k / k 2 d e s c r i b e d b y ( 3 0 )

i s m o r e t h a n a c o n v e x c o m b i n a t i o n o f t he v a r i a n c es

o f t h e i n d iv i d u a l t e r m s b e c a u s e o f t h e p r e s e n c e o f

t h e t e r m ( ~ k / k - - e i j) z - T i f fs s h o w s t h a t t h e v a r i a n c e

i s i n c r e a se d b y t h e p r e s e n c e o f t e rm s w h o s e m e a n

v a l u e s d i f f e r s i g n i fi c a n t l y f r o m t h e c o n d i t i o n a l m e a n

~ k / k. T h e i n f l u e n c e o f th e s e t e r m s i s t e m p e r e d b y

t h e w e i g h t i n g f a c t o r c ~ j . N o t e a l s o t h a t t h e c o n -

d i t i o n al v a r i a n c e ( i n c o n t r a s t t o t h e l i n e a r K a l m a n

f i lt e r ) i s a f u n c t i o n o f t h e m e a s u r e m e n t d a t a b e c a u s e

o f th e c i j an d th e (~k /k - - e i j ) .

T h e m e a n a n d v a r i a n c e o f t h e p r e d i c t io n d e n -

s i t y a r e d e s c r i b e d in a n o b v i o u s m a n n e r . I f t h e

g a u s s ia n s u m i s a n a p p r o x i m a t i o n t o t h e t r u e

n o i s e d e n s i t y , t h e s e r e l a t i o n s s u g g e s t t h e d e s i r a b i l i t y

o f m a t c h i n g t h e fi rs t t w o m o m e n t s e x a c t ly i n o r d e r

t o o b t a i n a n a c c u r a t e d e s c r i p t io n o f t h e c o n d i t i o n a l

m e a n ~ k+ l /k a n d v a r i a n c e P k + l /~ .

A s d i s c u s s e d e a r l i e r, i t is c o n v e n i e n t t o a s s i g n t h e

s a m e v a r i a n c e t o a l l t e r m s o f t h e g a u s s i a n s u m .

T h u s , i f t h e i n i ti a l s t a t e a n d t h e m e a s u r e m e n t a n d

n o i s e s e q u e n c e s a r e i d e n t i c a l l y d i s t r i b u t e d , i t i s

r e a s o n a b l e t o c o n s i d e r t h e v a r i a n c e s f o r a l l t e r m s t o

b e i d e n t i c a l a n d t o d e t e r m i n e t h e c o n s e q u e n c e s o f

t h is a s s um p t i o n . N o t e f r o m S c h o l i u m 3 . 2 t h a t i f

t h e n

0"i2 = O ' j 2 m o - 2

rU 2 = a 2 /2 f o r a l l i , j .

T h u s , t h e v a r i a n c e r e m a i n s t h e s a m e f o r a l l t e r m s

i n t h e g a u s s i a n s u m

w h e n p x k / Z k )

i s f o r m e d a n d t h e

c o n c e n t r a t i o n s a s d e s c r i b e d b y t h e v a r i a n c e b e c o m e s

g r e a t er . F u r t h e r m o r e , f r o m S c h o l i u m 3 .2 it f o l lo w s

t h a t t h e m e a n v a l u e i s gi v e n b y

i j - - ~ i 3 7 / ~ j

2

s o t h e n e w m e a n v a l u e i s th e a v e r a g e o f t h e p r e v i o u s

m e a n s . T h i s s u g g es t s t h e p o s s i b i l i t y t h a t t h e m e a n

v a l u e s o f s o m e t e r m s , s i n c e t h e y a r e t h e a v e r a g e o f

t w o o t h e r t e r m s , m a y b e c o m e e q u a l , o r a l m o s t

e q u a l. I f t w o t e r m s o f t he s u m h a v e e q u a l m e a n s

a n d v a r i a n c e s , t h e y c o u l d b e c o m b i n e d b y a d d i n g

t h e i r r e s p e c t iv e w e i g h t i n g f a c t o r s . T h i s w o u l d

r e d u c e t h e t o t a l n u m b e r o f te r m s i n t h e g a u s s ia n

s u m .

T h e

~j

i n ( 2 5 ) a r e e s s e n t i a l l y d e t e r m i n e d b y a

g a u s s i a n d e n s it y . T h u s , i f

Zk - -p ~ j

b e c o m e s v e r y

l a r g e , t h e n t h e c i j m a y b e c o m e s u f f i c i e n t l y s m a l l

t h a t t h e e n t i r e t e r m i s n e g l ig i b l e. I f t e r m s c o u l d b e

n e g l e ct e d , th e n t h e t o t a l n u m b e r o f t e r m s i n t h e

g a u s s i a n s u m c o u l d b e r e d u c e d .

T h e p r e d i c t i o n a n d f i l t e r i n g d e n s i t i e s a r e r e p r e -

s e n t e d a t e a c h s t a g e b y a g a u s s i a n s um . H o w e v e r ,

i t h a s b e e n s e e n t h a t t h e s u m s h a v e t h e c h a r a c t e r i s t i c

t h a t t h e n u m b e r o f t e r m s i n c r e a s e s a t e a c h s t a g e a s

t h e p r o d u c t o f th e n u m b e r o f te r m s i n t he t w o c o n -

s t i t u e n t s u m s f r o m w h i c h t h e d e n s i t i e s a r e f o r m e d .

T h i s f a c t c o u l d s e r i o u s l y r e d u c e t h e p r a c t i c a l i t y o

t h i s a p p r o x i m a t i o n i f t h e r e w e r e n o a l l e v i a t i n g

c i r c u m s t a n c e s . T h e d i s c u s si o n a b o v e r e g a r d i n g t h e

d i m i n i s h i n g o f t h e w e i g h t i n g f a c t o r s a n d t h e c o m -

b i n i ng o f t e r m s w i t h n e a r l y e q u a l m o m e n t s h a s

i n t r o d u c e d t h e m e c h a n i s m s w h i c h s i g n i f i c a n t l y

r e d u c e t h e a p p a r e n t i ll e f fe c t s c a u s e d b y t h e i n c r e a se

i n t h e n u m b e r o f t e r m s i n th e s u m . I t is a n o b s e r v e d

f a c t t h a t t h e m e c h a n i s m s w h e r e b y t e r m s c a n b e

n e g l e ct e d o r c o m b i n e d a r e i n d e e d o p e r a t i v e a n d i n

f a c t c a n s o m e t i m e s p e r m i t t h e n u m b e r o f t e r m s i n

t h e s e r ie s t o b e r e d u c e d b y a s u b s t a n t i a l a m o u n t .

S i n c e w e i g h t i n g f a c t o r s f o r i n d i v i d u a l t e r m s d o

n o t v a n i s h i d e n t i ca l l y n o r d o t h e m e a n a n d v a r i a n c e

o f m o r e g a u s s i a n d e n s i t i e s b e c o m e i d e n t i c a l, i t is

n e c e s s a r y t o e s t a b l i s h c r i t e r i a b y w h i c h o n e c a n

d e t e r m i n e w h e n t e r m s a r e n e g l ig i b l e o r a r e a p p r o x i -

m a t e l y t h e sa m e . T h i s is a c c o m p l i s h e d b y d e fi n i n g

n u m e r i c a l t h r e s h o l d s w h i c h a r e p r e s c r i b e d t o m a i n -

r a i n t h e n u m e r i c a l e r r o r le s s t h a n a c c e p t a b l e l i m i t s.

C o n s i d e r t h e e f f e c t s o n t h e L ~ e r r o r o f n e g l e c t i n g

t e r m s w i t h s m a l l w e i g h t i n g f a c t o r s . S u p p o s e t h a t

t h e d e n s i t y i s

p ( x ) = ~ o q N a x - a i ) (3 3 )

i = 1

a n d s u c h t h a t a l , a 2 . . . . , ~ , , - l ( m < n ) a r e l es s t h a n

s o m e p o s i ti v e n u m b e r 6 ~ . N o t e t h a t t h e v a r ia n c e

h a s b e e n a s s u m e d t o b e t h e s a m e f o r e a c h t e r m .

C o n s i d e r r e p l a c i n g p b y P a w h e r e

pA(X) = 1 ~

o~iN, , x - -a i ) .

(3 4 )

~ , O ~i i = m

i = m

T h e t o l l o w i n g

d i f f icu l ty .

T h e o r e m

3.4.

b o u n d i s

d e t e r m i n e d w i t h o u t

f

~ m - - 1

Ip(x)-p (x)ldx 2 E ( 3 5 )

1 = 1

< 2(m - 1 )61 . (36)

T h e L ~ e r r o r c a u s e d b y n e g l ec t i ng ( m - 1) t e r m s

each o f w h ich a r e l e s s th an f i~ i s s een in (3 5 ) to b e

l e s s t h a n t w i c e t h e s u m o f t h e n e g l e c t e d t e r m s .

T h u s , t h e t h r e s h o l d fi~ c a n b e s e l e c t e d b y u s i n g ( 3 6 )

o r ( 3 5 ) t o k e e p t h e i n c r e a s e d L 1 e r r o r w i t h i n

a c c e p t a b l e l i m i ts .

C o n s i d e r t h e s i t u a t i o n i n w h i c h t h e a b s o l u t e

v a l u e o f t h e d i f f e r e n c e o f t h e m e a n v a l u e s o f t w o

t e r m s i s sm a l l . I n p a r t i c u l a r , s u p p o s e t h a t a 1 a n d a 2

a r e a p p r o x i m a t e l y t h e s a m e a n d c o n s i d e r t h e L 1

e r r o r t h a t r e s u l t s i f t h e

p x )

g iv en in (3 3 ) i s r ep lac ed

b y

p a ( x ) = ~

o q N , x - a i ) + o q + o t z ) N ~ x - ~ )

(3 7 )

i = 3


11/15


w h e r e

U s i n g ( 3 7 ) , o n e c a n p r o v e t h e f o l l o w i n g b o u n d .

F o r a d e t a i l e d p r o o f o f th i s a n d o t h e r r e s u l t s , se e

Ref. [17].

Theorem 3.5.

f ~ _ o l P x ) - p A X ) l d x < _ 4 ~ 2 M l a 2 - a l I+ ~

(3 8 )

T h u s t e r m s c a n b e c o m b i n e d i f t h e r i g h t - h a n d s i d e

o f ( 3 8 ) i s l e s s t h a n s o m e p o s i t i v e n u m b e r 6 2 w h i c h

r e p r e s e n t s t h e a l l o w a b l e L ~ e r r o r . T h e M i n ( 3 8) i s

t h e m a x i m u m v a l u e o f N , a n d i s gi v e n b y

1

M =

O b s e r v e t h a t a s t h e v a r i a n c e t r d e c r e a s e s , t h e

d i s t a n c e b e t w e e n t w o t e r m s t o b e c o m b i n e d m u s t

a l s o d e c r e a s e i n o r d e r t o r e t a i n t h e s a m e e r r o r

b o u n d .

3.2. A numerical example

I n t h i s s e c t i o n t h e r e s u l t s p r e s e n t e d i n s e c t i o n 3 .1

a r e a p p l i e d t o a s p e c i fi c e x a m p l e . T o m a k e ( 1 7 )

a n d ( 1 8 ) m o r e s p e c i f ic , s u p p o s e t h a t t h e s y s t e m i s

X k ~ - - X k - I ~ Wk- 1

(3 9 )

Zk = Xk + vk

(4 0 )

w h e r e t h e xo, Wk, Vk k= O , 1 . . . ) a r e a s s u m e d t o

b e u n i f o r m l y d is t r i b u te d o n ( - 2 , 2 ) as d e fi n e d b y

(15).

T h e p r o b l e m t h a t i s c o n s i d e r e d re p r e s e nt s s o m e -

t h i n g o f a w o r s t c a s e f o r t h e a p p r o x i m a t i o n b e c a u s e

t h e i n i ti a l s t a t e a n d t h e n o i s e s e q u e n c e s a r e a s s u m e d

t o b e u n i f o r m l y d i s t r i b u t e d w i t h t h e d e n s i t y d i s -

c u s s e d i n s e c t i o n 2 .2 . A s d i s c u s s e d t h e r e , t h e d i s -

c o n t i n u i t i e s a t + 2 m a k e i t d i i f ic u l t t o f i t t h i s d e n s i t y

a n d n e c e s si t at e s th e u s e o f m a n y t e r m s i n t h e

g a u s s i a n s u m . T h e s p e c if i c a p p r o x i m a t i o n u s e d

h e r e c o n t a i n s 1 0 t e r m s a n d i s s h o w n i n F i g . l ( b ) .

I t i s a p p a r e n t t h a t t h i s a p p r o x i m a t i o n h a s n o n -

t r i v i a l e r r o r s i n t h e n e i g h b o r h o o d o f t h e d is -

c o n t i n u i t i e s b u t n o n e t h e l e s s r e t a i n s t h e b a s i c

c h a r a c t e r o f t h e u n i f o r m d i s t ri b u t io n .

T h e p e r f o r m a n c e o f th e g a u s si a n s u m a p p r o x i m a -

t i o n f o r t h i s e x a m p l e i s d e s c r i b e d b e l o w b y c o m -

p a r i n g t h e c o n d i t io n a l m e a n a n d v a r i a n c e p r o v i d e d

b y t h e a p p r o x i m a t i o n w i t h t h a t p r e d i c t e d b y t h e

K a l m a n f i l t e r a n d w i t h t h e s t a ti s t ic s o b t a i n e d b y

c o n s i d e r in g t h e t r u e u n i f o r m d i s t ri b u t i on . T h e

l a t t e r h a v e b e e n d e t e r m i n e d f o r t h is e x a m p l e a f t e r a

n o n t r i v ia l a m o u n t o f n u m e r i c a l c o m p u t a t i o n . I n

a d d i t i o n t h e t r u e a posteriori d e n s i t y p Xk/Zk) h a s

b e e n c o m p u t e d a n d i s c o m p a r e d w i t h t h a t o b t a i n e d

u s i n g t h e a p p r o x i m a t i o n .

T h e K a l m a n e r r o r v a r i a n c e is i n d e p e n d e n t o f t h e

m e a s u r e m e n t s e q u e n c e a n d c a n c a u s e m i s l e a d i n g

f i l te r r e s p o n s e . F o r e x a m p l e , f o r s o m e m e a s u r e -

m e n t s e q u e n c e s p e r f e c t k n o w l e d g e o f t h e s t a t e i s

p o s s i b l e , t h a t i s t h e v a r i a n c e i s z e r o , b u t t h e K a l m a n

v a r i a n c e s t il l p r e d i c t s a l a r g e u n c e r t a i n t y . F o r

e x a m p l e , s u p p o s e t h a t t h e m e a s u r e m e n t s a t e a c h

s t a g e a r e e q u a l t o

Z k = 2 k + 4 , k = 0 , 1 . . .

T h e n , t h e m i n i m u m m e a n - s q u a r e e s t i m a t e f o r t h e

s t a t e b a s e d o n t h e u n i f o r m d i s t r i b u t i o n i s

. k / k = 2 k + 2 , k = O , 1 . . .

a n d t h e v a r i a n c e o f t h i s e s t i m a t e i s

Pk/k2=O f o r a l l k .

T h u s , f o r t h i s m e a s u r e m e n t r e a l i z a t i o n t h e m i n i -

m u m m e a n - s q u a r e e s t i m a t e is e r r o r - f r e e . S i n c e t h e

K a l m a n v a r i a n c e is in d e p e n d e n t o f t h e m e a s u r e -

m e n t s , i t is n e c e s s a ri l y a p o o r a p p r o x i m a t i o n o f t h e

a c t u a l c o n d i t i o n a l v a r i an c e . T h e s q u a r e r o o t o f t h e

K a l m a n v a r i a n c e 7KAL a n d t h e e r r o r i n t h e b e s t

l i n e a r e s ti m a t e t o g e t h e r w i t h t h e s q u a r e r o o t o f th e

v a r i a n c e a~s a n d e r r o r i n t h e e s t i m a t e o f t h e s t a te

f o r t h e t e n t e r m g a u s s ia n s u m f o r t hi s m e a s u r e m e n t

r e a l i z a t i o n is s h o w n i n F i g . 3 (a ) . T h i s s h o w s t h a t a

c o n s i d e r a b l e i m p r o v e m e n t i n b o t h t h e m e a n a n d

v a r i a n c e i s p r o v i d e d b y t h e g a u s s i a n s u m w h e n

c o m p a r e d w i t h t h e r e su l ts p r o v i d e d b y th e K a l m a n

f i l ter .

(a)

o ' i o

0 8

0-7

0.6

0 5

0 4

0-5

0.2

0 1

%

I 0

0-9

0.8

0.7

0.6

0.5

0-4-

0 .3

0.2

0.1

0 0

+ f . t r - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + C T K A

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ~ K A L

. . . . '

f

/

( .

: 1 - ~ . . . . . . . . . X C S

- / .

v I 1 I I I I I I 1 I I I I 1 I I I I I

I 2 3 4 5 6 7 8 9 10 II 12 13 14 15 16 17 18 19 20

K

b )

I / /

I , , ' / V , x o o ,

~ . a r R u z

- 5 ', "~'.-'.,:'-~c.-~'


12/15

476 H .W . SORENSON and D. L. ALSPACH

The measurement seq uence d escr ibed above i s

highly imp robab le. A more representat ive case is

d ep ic t ed in F ig . 3 (b ) in w h ich t he p lant and measure-

ment noise s eq uences were chosen us ing a rand om

number generat or for t he un i form d is t r ibut ion

prescr ibed above . Th is f igure again show s t hat t he

gauss ian sum approx imat ion t o t he t rue var iance i s

cons id erab ly improved over t he Kalman es t imat e

and, in fact , agrees very closely with the true

s t and ard d eviat ion a T RU ~ of t he

p o s t e r i o r i

density.

However , t he Kalman var iance i s represent at ive

enough t hat t he error in t he es t imat e of t he mean i s

not part icularly d if ferent than that provided by the

gaussian sum.

T h e p o s t e r i o r i density funct ion for the third and

fourteenth stages is shown in Figs . 4 (a) and 4(b) .

In these f igures , the actual density, the gaussian

approx imat ion provid ed by t he Kalman f i l t er and

t he gauss ian sum approx imat ion are a l l inc lud ed .

At stage 3 , the true variance is smal ler than the

Kalman var iance and t he Kalman mean has a

s ignif icant error whereas at s tage 14 the true

variance is larger than that predicted by the Ka lman

fi lter . No te that the error in the

p r i o r i

density

approximat ion at the d iscont inuit ies is s t i l l evident

in the

p o s t e r i o r i

approx imat ion but t hat t he

general character of the density is reproduced by

the gaussian sum.

b .

( : 3

13.

0 , 7 5

0 . 5 0

0 2 5

a )

T r u e p r o b a b i l i t y d e n s i t y

. . . . . . . K a l m a n a p p r o x i m a t io n

. . . . G a u s s i a n s u m a p p r o x i m a t io n

/ \

l /

+ . . \

. 7 .

. : . ' '. + . \

. f'/ / .

. . . i + . . ' \

\

' '#

+ ~ . X

, , , / / 't..

\

i i

0 5 I ' 0 1 ' 5 2 0 2 ' 5 S ' O

x

0 . 5

0 . 4

~ 0 . 3

a

( 3 -

0 2

O i

I 0

b )

. . . . ,

, . . \

. . L

' ,

j .:.:,',,,

/ ," ". \

/ ' .. \

. : . .. \ \

/ . : ' . \

/ . . ' ' ,

/ . . ' ' .

1 , 0 2 ' 0 3 - 0 . 4 - 0

x

~ 3

5 0

0 ' 5 0

r ~

Q .

0 ' 2 5

( c )

o .7 5 T r u e p r o b a b i l i t y d e n s i t y

. . . . . 8 ~ = 0 . 0 0 1 = 8

. , . ' , ~

; ' ~

~ I I I

0 - 0 0 . 5

0 . 4 -

0 3

Q

n O . 2

%

I I 0

I ' 0 I - 5 2 0 2 ' 5 S ' O 0

x

d )

. .. .. .. .. .. 8 , = 0 . 0 O I , 8 = 0 . 0 0 . 5

. . . . . 8 1= 0 0 0 5 , 8 2 = 0 . 0 O I

. ~ . ~ :+ -h +

Y

;W

~ ..

/ . ' - I : \

~ - 1 , , , , , , ,

1 . 0 2 0 3 0 4 . 0

x

FIO. 4. Influence of 61, &z on p o s t e r i o r l density.

(a) Third stal~l---82------0 001.

Co) Fourteenth stag~---Sl =8 2 =0 .00 1.

( c) T hir d s t a l l = 0 0 0 1 , 6 2 = 0 . 0 0 5 . a n d 5 t = 0 0 0 5 , 6 1 = 0 0 0 1 .

(d) Fourteenth staile---alffi04)05, &z=0 001. and 51 =0 .00 1, 8 z= 0.0 05 .

\

5 ' 0


13/15

Recursive Bayesian estimation using gaussian sums 477

In Figs. 4(a) and 4(b) the approximations at each

stage are based on terms being eliminated when

their weighting functions are less than St=0.001

and combined when the difference in mean values

cause the L ~ error to be less than 52=0.001. The

effect of changing these parameters, that is, 5~ and

52, can be seen in Figs. 4(c ) and 4(d). In these

figures, the effect of changing 5t and c52 is shown.

In one case tS~ is increased from 0.001 to 0.005 while

keeping 52 equal to 0.001. Alternatively, 52 is in-

creased to 0.005 while the value of 5~ is maintained

equal to 0.001. It is apparent in this example tha t

52 has a larger effect on the density approximation.

The increase in this parameter can be seen to

introduce a ripple into the density function and

indicates that the individual terms have become too

widely separated to provide a smooth approxima-

tion. It is interesting tha t the effect appears to be

cumulative as the ripple is not apparent after three

stages but is quite marked a t the 14th stage.

The effect of improving the accuracy of the

density approximation by including more terms in

the

a p r i o r i

representations and by retaining more

terms for the


representation can be seen

in Fig. 5. Twenty terms are included in the

a p r i o r i

densities and 5~ and 52 are reduced to 0.0001. The

figure presents the actual


density, the

gaussian sum approximation, and the gaussian

approximation provided by the Kalman filter

equations. Comparison of the results for stages 3

and 14 with Fig. 4 indicates the improvements in

the approximation that have occurred.

- - T r u e pr o b a bi l it y d e n si ty

. . . . . Kalma n approximation

G a u s s i a n s u m ap p r ox i ma t i on

4 . C O N C L U S I O N S

The approximation of density functions by a

sum of gaussian densities has been discussed as a

reasonable framework within which estimation

policies for non-linear and/or nongaussian stochastic

systems can be established. It has been shown that

a probability density function can be approximated

arbitrarily closely except at discontinuities by such

a gaussian sum. In contrast with the Edgeworth or

Gram-Charlier expansions that have been in-

vestigated earlier, this approximation has the

advantage of converging to a broader class of

density functions. Furthermore, any finite sum of

these terms is itself a valid density function.

The gaussian sum approximation is a departure

from more classical approximation techniques

because the sum is restricted to be positive for all

possible values of the independent variable. As a

result, the series is not orthogonalizable so tha t the

manner in which parameters appearing in the sum

are chosen is not obvious. Two numerical pro-

cedures are discussed in which certain parameters

are chosen to satisfy constraints or are somewhat

arbitrarily selected and others are chosen to mini-

mize the

k

error.

It is anticipated that the gaussian sum approxima-

tion will find its greatest application in developing

estimation policies for non-linear stochastic systems.

However, many of the characteristics exhibited by

non-linear systems and some of the difficulties in

using the gaussian sum in these cases are exhibited

by treating linear systems with nongaussian noise

0 5 0 F . . 2 O F 0 . 7 5 I - 0 ' 5 0 I -

L l b . . l . . . . .

. . . . u _ - - - ' '

02 5 ~ . - - '~ - "m-- -~- I o I0 c -, . . a .- 0 25

/ . . . . . . . . . . . . . . . . . .

_

o , : . , o l . . . . . ; i o o - i o [ . ' ' ,

-~

- 2 - 5 0 0 2 5 0 2 0 1 .5 0 5 ' 0 0 0 2 . 5 0 5 . 0 0

x X X X

0 . 5 0 F . . . . I 0 - 0 - 5 O f - 0 5 0 [ - . ' . .

. . . . . t - b - , , t -

0 .~ :5 . .' ~ _ , ~ l 1 . ~ % 2 5 t - . .7 . t ~ c ~ t - I ,' . . ' ' - '- ' ~ , , ,

o , . . . , J , ' l _ , o l . . . . . . . . . : / , , I % o 1 . ; ' 1 , ,

- I 0 0 i 5 0 - 2 5 0 ' , 0 0 5 0 0 0 2 . 5 0 5 0 0 1 .5 0 4 - 0 0 6 - 5 0

X X X X

0 . 5 0[ 0 5 0 [ 0 5 0 l I 0 [

1 ' 50 4 ' 0 0 6 ' 5 0 0 2 5 0 0 0 1 .5 0 4 0 0 2 0 4 0 6 0

X X X X

5 r l - . . --~-'~m-q/ k 5 r 5 1 - I f ~

o L , ~ f 1 I I I f ' . , o l , . d " ~ I

r

r - . , , o L. ,L " ~ I I I t z ' ~ oi ~ i i t i" L'~---.t

0 2 5 0 5 0 0 0 2 5 0 5 . 0 0 C ,'~ O 2 . 0 0 4 - 5 0 i O 4 0 5 . 0. 6 0

X X X X

Fie 5 A p o s te r io r i d e n s i t y f o r s i x t e e n s t a g e s .


14/15

478 H . W. SORENSON an d D. L. ALSPACH

sources. Of course, whe n the noise is entirely

gaussian the problem degenerates and the familiar

Kalman filter equations are obtained as the exact

solut ion of the problem.

The l inear, nongauss ian est imation problem is

discussed and it is shown that, if the

a priori

density

functions are represented as gaussian sums, then

the number of terms required to describe the a

posteriori

density is equal to the product of the

number of terms of the

a priori

densities used to

form it. The appar ent disa dvanta ge is seen to cause

little difficulty however, b ecause the m omen ts of

many individual terms converge to com mon values

which allows them to be combi ned. Furth er, the

weighting factors associated with man y other terms

become very small and permit those terms to be

neglected without introducing significant error to

the approximation.

Numerical results for a specific system are

presented which provide a demonstrat ion of some

of the effects discussed in the text. These results

confirm dramatical ly that the gaussian sum approxi-

mation can provide considerable insight into

problems that hi therto have been intractable to

analysis.

REFERENCES

[1] A. H. JAZWINSKI: Stochastic Processes and Filtering

Theory.

Academic Press, New York (1970).

[2] R. E. KALMAN: A new approach to l inear filtering and

prediction problems.

J. bas. Engng

82D, 35--45 (1960).

[3] H. W. SORENSON

Advances in Control Systems

Vol. 3,

Ch. 5. Academic Press, New York (1966).

[4] R. COSAERT and E. GOTTZEIN: A decoupled shifting

memory filter method for radio tracking of space

vehicles. 18th International Astronautical Congress,

Belgrade, Yugoslavia (1967).

[5] A. JAZWINSKI: Adaptive filtering. Automatica 5 475-

485 (1969).

[6] Y. C. Ho and R. C. K. LEE: A Bayesian approach to

problems in stochastic estimation and control.

IEEE

Trans. Aut. Control

9, 333-339 (1964).

[71 H. J. KUSHNER: On the differential equations satisfied

by conditional probability densities of Markov pro-

cesses.

SIAMJ. Control

2, 106-119 (1964).

[8] J. R. FmrlER and E. B. STEAR: Optimal non- linear

filtering for independent increment processes, Parts I

and II.

IEEE Trans. Inform. Theory

3, 558-578 (1967).

[9] M. AOKI:

Optimization of Stochastic Systems Topics

in Discrete-Time Systems.

Academic Press, New York

(1967).

[10] H. W. SORENSON and A. R. STOBBERUD: Nonlinear

filtering by approximation of the

a posteriori

density.

Int. J. Control 18, 33-51 (1968).

[11] M. AOKI: Optimal Bayesian and rain-max control of a

class of stochastic and adaptive dynamic systems.

Pro-

ceedings IFA C Symposium on Systems Engineering for

Control System Design

Tokyo, pp. 77-84 (1965).

[12] A. V. CAMERON: Control and estimation of linear

systems with nongaussian

a priori

distributions.

Pro-

ceedings of the Third Annual Conference on Circuit and

System Science

(1969).

[13] J. T. Lo: Finite dimensional sensor orbits and optimal

nonl inear filtering. University of Southern California.

Report USCAE 114, August 1969.

[14] J. KOREYAAR: Mathematical Methods Vol. 1, pp. 330-

333. Academic Press, New York (1968).

[l 5] W. FELLER: An Introduction to Probability Theory and

Its Applications

Vol. lI, p. 249. John Wiley, New York

(1966).

[16] J. D. SI'RAOINS: Reproducing distributions lbr machine

learning. Standard Electronics Laboratories, Technical

Report No. 6103-7 (November 1963).

[17] D. L. ALST'ACH: A Bayesian approximation technique

for estimation and control of time-discrete stochastic

systems. Ph.D. Dissertation, University of Cahfornia,

San Diego (1970).

R6sum~--Les relations recurrentes bayesiennes qui decrivent

le comportement de la fonction de densit6 de probabilit6

a posteriori de l'6 tat d'un syst6me al6atoire, discret dans le

temps, en se basant sur les donn6es des mesures disponibles,

ne peuvent ~tre g6n6ralement resolues sous une forme fermee

lorsque le syst6me est soit non-lin6aire, soit non-gaussien.

ke present article introduit et propose une approximation de

densit6, mettant en jeu des combinaisons convexes de fonc-

tions de densit6 gaussiennes, ~t titre de m6thode efficace pour

6viter les difficultes rencontr6es dans l'6valuat ion de ces

relations et dans l'utilisation des densit6s en r6sultant pour

determiner des strat6gies d'6valuation particuli6res. II est

montr6 que, lorsque le nombre de termes de la somme

gaussienne augmente sans limites, l'approximation converge

uniformement vers une fonction de densit6 quelconque dans

une cat6gorie 6tendue. De plus, toute somme finie est elle-

m6me une fonetion de densit6 valable, d'une mani6re

diff6rente des nombreuses autres approximations 6tudi6es

dans le pass6.

Le probl6me de la determination des estimations de la

densit6 a posteriori et de la variance minimale pour des

syst~mes lin~aires avee bruit non-gaussien est trait6 en

utilisant l'approximation des sommes gaussiennes. Ce

probl6me est 6tudi6 parce qu 'il peut ~tre trait6 d'une mani~re

relativement simple en utilisant l'approximation mais

cont ient encore la plupart des difficult~.s rencontr6es en

consid6rant des syst+mes non-lin6aires, puisque la densit6

a posteriori est non-gaussienne. Apr6s la discussion du

probl6me g6n6ral du po int de vue de l'applicationdes sommes

gaussiennes, l'article pr6sente un exemple num6rique dans

lequel les statistiques r6611es de la densit6

a posteriori

sont

compar6es avec les valeurs pr6dites par les approximations

des sommes gaussiennes et du filtre de Kalman.

Zusammenfassung--Die Bayes'schen Rekursionsbezie-

hungen, die das Verhalten der a priori-Wahrscheinlichkeits-

dichtefunktion des Zustandes eines zeitdiskreten stochasti-

schen Systems beschrieben, k6nnen nicht allgemein in

geschlossener Form gel6st werden, wenn das System ent-

weder nichtlinear oder nicht-gaussisch ist. In dieser Arbeit

wird eine Dichteapproximation, die konvexe Kombina-

tionen von Gauss'schen Dichtefunktionen enth~ilt, eingefiihrt

und als bedeutungsvoUer Weg zur Umgehung der Schwierig-

keiten vorgeschlagen, die sich bei der Auswertung dieser

Beziehungen und bei der Benutzung der resultierenden

Dichten ergeben, um die spezifische Schiitzstrategie zu

bestimmen. Ersichtlich konvergiert, da die Zahl der Terme

in der Gauss'schen Summe unbegrenzt w~ichst, die Approxi-

mation gleichm~il~igzu einer Dichtefunktion in einer um-

fassenden Klasse. Weiter ist eine endliche Summe selbst eine

gfiltige Dichtefunktion, anders als viele andere schon

untersuchte Approximationen.

Das Problem der Bestimmung der Sch~ttzungen der a

posteriori-Dichte und des Varianzminimums ftir lineare

Systeme mit nichtgauss'schem Ger~iusch wird unter Behand-

lung der Gauss'schen Summenapproximation behandelt.

Dieses Problem wird betrachtet, well es in einer relativ

geraden Art unter Benutzung der Approximation behandelt

werden kann, abet immer noch die meisten der Schwierig-

keiten enth~ilt, denen man bei der Betrachtung nichtlinearer

Systeme begegnet, da die a posteriori-Dichte nichtgaussisch

ist. Nach der Diskussion des allgemeinen Problems vom

Gesichtspunkt der Anwendung Gauss'scher Summen, wird

ein Zahlenbeispiel gebracht, in dem die aktuellen Statistiken

der a posteriori-Dichte mit den Werten verglichen werde,

die durch die Gauss'sche Summe und durch die Kalman-

Filter-Approximationenvorhergesagt wurden.


15/15

Recursive Bayesian estimation using gaussian sums 479

Pe~IOMe---PeKypcHBHMe

6a~eco m : r ~e

BMpa~eHH~ OnHCM-

aalottlne nOBelIeHHe anocTepHopHoi~ ~ymmH nnOTHOCTH

BepO~ITHOCTH COCTOIIHH~Icay~aI~o~ CHCTeMbI, ]IHCI{peTHO~

n o

SpeMetlrl, OCHOSl,magcl, Ha ~ocTynm~Ix pe3ynbTaTax

H3MepeHgl~, He M O F y T 6MTI, O61,DIHO peii ieHl , B 3aMKHyTOI~

dpOpMe Kor ea CHCTeMa HH60 HCHHHe]~Ha, dIH6 0-~e He

ssnaerc~ raycco~croit HaCTOSi~a~

CTaTb~ BBO~HT H

npemmraer

npH6HH~KeHHe ILrlOTHOCTH,

npa6craromcc

K

COqeTallI~lM

ayccoBCKHX dpyHKl1Hlt HHOTHOCTH, B Ka~ICCTBe

3Op~eKTHBHOFO MeTo~Ia ~n a H3~KaHHfl 3aTpy~HeHHl~

BCTpeqaeMblX B OI~eHKe 3THX Bblpa~KeHH~ H B HCHO.rlb3OBaHHH

B],~TeKaIOILmX H3 HHX HJIOTHOCTeI'~ ~II~1 onpe~eHH~ ~aHHblX

CTaTerH.fi OReHKH. I'IoKa3MBaeTC~ qTO, KOFRa ~IHCHO

~IeHOB raycco~croit CyMMbI 6ecnpe~eH~HO Bo3pacTaeT,

rlpH6nH~KeHHe paBHOMepHo CTpeMrlTC~I K n/o6 o~ t (I~yHKI~HH

ILrIOTHOCTH B IIIHpOKOM KJlaCC~. CB ep x ~TOFO, BCaKa~I

KOHeqHall CyMMa caM a ~IBJl~leTCfl rlpHeMneMOl'~ ~yHKHHeI~

IIHOTHOCTH~ B OTJIPIqI4H OT MHOFO~4~HCHeHH]bIX ~Ip yr tl x

npH6Jm~cermll H3y~aBmHXC~ S np omn oM.

I-[po6J1eMa oHpo~eJ1erm~ OHOHOK anOCTOpHOpHOfl rmO-

THOCTII H MHHHMaYlbHO~ BapHaHTHOCTH ~flH HHHCflHbIX

CHCTeM C IIo-rayccoBCIOIM IIIyMOM pemaeTcf l c n oMo tI~r O

npH6nrmcon~ rayccoBcrdax CyMM. ~Ta npo6fleMa n3y-

tlaeTCfl IIOTOMy qTO OHa MO)KeT 6bITb peI iie Ha cpaBHll Te-

JlbHO npOCTO Hcno.rtl,3y~ n pH fm , xcerme HO co~lep~crlT e me

60JIBIIIHHCTBO 3aTpy~IHeHni~ BCTpeqaeMI, IX pacCMaTpHBa~

HeaaHeianbie CHCTeMbI, H60 ario crepa opria a nn or ao cr b

aa ~e wc s He-raycc oBczoit Hoc ne o6cy:~cjXeHaa o6IUeR

npo6H eMb l C TOqKH 3peHHll HpHMeHeHH~I rayCCOBCKHX

CyMM~ CTaTl~fl ~laeT qHCJIOBOI~ rlpltMep B KOTOpOM ~el~CTBHTe-

IlbHble CTaTHCTHKH anO cTepr lopHo lt ILrlOTHOCTH cpaBHHBalO-

TCfl C 3HaLIeHH~VIH Hpe~cKa3at-IHblMrl HpH6HH)KeHri~Mri

rayc coac Kax CyMM H ~HHbTpa KanMana.

recursive bayesian estimation using gaussian sums

Documents