Two Dimensional Affine Projection Adaptive Filter Algorithms with Low Computational Complexity

Features

Mohammad Shams Esfand Abadi, and Sahar Nikbakht Aali

  

Abstract-Two-dimensional (2D) adaptive filtering is a technique which is applicable to many image, and signal processing applications. This paper extends one-dimensional adaptive filter algorithms to their corresponding 2D counterparts. Based on this extension, a 2D selective partial update APA (2D-SPU-APA) and 2D partial rank affine projection (2D-PRA) are presented. Then, by the combination of these two algorithms, a new 2D adaptive filter which is called 2D selective partial rank affine projection algorithm (2D-SPR-APA) is introduced. We demonstrate the good performance and applicability of our proposed algorithms by conducting several simulations in the applications of 2D system identification and noise cancellation.

Keywords-Two dimensional adaptive filters, selective

partial update method, partial rank affine projection algorithm, 2D system identification.

I. INTRODUCTION

WO-dimensional (2D) adaptive filters have received a great deal of attention in the last two decades. The main reason for this attention is the ability of these adaptive

filters to take into account the inherent nonstationary statistical properties of two dimensional data. In [1], the one dimensional least mean squares (LMS) adaptive filtering algorithm was extended to the 2D applications. The 2D-APA has been proposed in [2] which its performance on input signals with high correlation was excellent. A fast 2D affine projection was proposed in [3]. Although the 2D version of the RLS algorithm is also proposed [4, 5], but because of its high computational complexity, its usage is not recommended in 2D applications. In classical adaptive filters, usually the entire coefficients of the filter are updated, simultaneously. However, to reduce the computational complexity of the corresponding updating rules, in applications dealing with one dimensional signals, some refined version of these filtering algorithms are also introduced. In these algorithms, at each iteration, only a subset of filter coefficients is updated.

Mohammad. Shams Esfand Abadi is with the  Faculty of Electrical and Computer Engineering, Shahid Rajaee Teacher Training University, Tehran, Iran. e-mail: mshams@srttu.edu Sahar. Nikbakht Aali is with the Faculty of Electrical and Computer Engineering, Shahid Rajaee University, Tehran, Iran. e-mail: sr_nikbakht@yahoo.com

The Max-NLMS [6], SPU-APA [7], selective regressor APA (SR-APA) [8] and the selective partial update subband adaptive filter (SPU-SAF) [9] are some of important examples of these adaptive algorithms that belong to this category.

In this paper, we extend two of the most important approaches to reduce the computational complexity of conventional 2D -APA to establish of the 2D-SPU-APA and 2D-PRA. Then we combine these algorithms to create 2D selective partial rank affine projection (2D-SPR-APA) filtering method which has low computational complexity. We will demonstrate the performance of our proposed algorithms on different 2D applications such as 2D system identification and noise cancellation.

The paper is organized as follows: In Section 2 our novel 2D adaptive filter algorithms are presented. Computational complexity of the newly developed adaptive filtering methods is computed in Section 3. Section 4 is devoted to the presentation of several simulation results performed in the fields of 2D system identification and noise cancellation (for the elimination of noise in digital images).

Throughout the paper the following notations are adopted:

(.)T Transpose of vector or a matrix

Tr(.) Trace of a matrix 2

. Square Euclidean norm of a vector

F. Frobenius norm

.E Expectation operator

II. PROPOSED ALGORITHM

A. 2D-SPU-APA The update equation for 2D-APA can be written as follows

[3].

( , ) = ( , ) +μ ( , ) (1)k +1 k ki j i j i jw w U

T -1( ( , ) ( , ) + δ ) ( , )k k ki j i j i jI U U e  

where in this equation,  μ  is the step size that controls the rate

of convergence,  k  is the iteration index, δ  is a positive small

number added to keep T ( , ) ( , ) k ki j i jU U singular. Also,

( , )wk i j and ( , )k i jU are weight vector of the filter and input

matrix respectively defined as follows

T

International Conference on Electronics, Biomedical Engineering and its Applications (ICEBEA'2012) Jan. 7-8, 2012 Dubai

244

2( , ) = [ (0,0) (0,1)... (0, -1) (2)w k i j w w w N

 T

1 2(1,0)... ( -1, -1)]w N N ˆ ˆ ˆ( , ) = [ ( , ) ( -1, )... ( +1, )] (3)U U U Uk k k ki j i j i j i - L j

where 1N and  2N describe the order of the filter and

ˆ ( , )U k i j can be written for = 0,1,... -1m L , based on input

vector, i.e ( , )uk i j as ˆ ( , ) = [ ( , ) ( , -1). (4)U u uk k ki - m j i - m j i - m j

k.. ( , +1)]i - m j - Ku      

2( , ) = [ ( , ) ( , -1)... ( , - +1) (5)uk i j u i j u i j u i j NT

1 2( -1, )... ( - +1, - +1)]u i j u i N j N   In addition, in “(1),” ( , )ek i j is the error signal which can

be expressed as

( , ) = ( , ) - ( , ) ( , )Tk k k ki j i j i j i je d U w                                        (6)

where, ( , )k i jd is the desired vector which is equal to

( ) [ ( ) ( 1) ( 1)k i, j = d i, j d i, j - ...d i, j - L+d  

T( -1, )... ( +1, +1)]d i j d i - K j - L                             (7)

As it can be seen in above equations as well, 2D-APA filter use ( )KL number of blocks to updates weight

coefficients of the filter. Parameters 10 K N and

20 L N  are often chosen to be smaller than 1N  and 2N ,

so the computational complexity of the algorithm remains low. From these aforementioned equations, it can be simply

inferred that the size of input matrix is 1 2( )N N KL , while the

size of error and desired signals is ( 1)KL .

To reduce the computational complexity of the conventional 2D-APA, we introduce the 2D-SPU-APA. In this novel algorithm the input matrix is partitioned into 1N blocks, each

with length 2( × )N KL , and then in each iteration only S  blocks of these 1N blocks is use to update weight matrix of the

filter. Consequently, we can write the filter coefficients update equation for 2D-SPU-APA as

T -1

( , ) = ( , ) + ( , )

( ( , ) ( , ) + δ ) ( , )

k+1 k k k

k k k k

i j i j i j

i j i j i j

I

w w A U

U A U e                        

(8)

where the kA is the 1 2 1 2( × )N N N N  diagonal matrix with the

1 and 0 blocks each of length 2N on the diagonal where  the

positions of 1’s on the diagonal determine which coefficients of the filter should be updated at each iteration. The only question that we should answer is how to determine matrix kA

in each iteration. For this purpose, let us at first define new

matrix namely ˆ ( , )kk i jU with the size of 2( )N KL   for any  

10 ( -1)k N as  

ˆ ˆ ˆ ˆ ˆ( , ) = [ ( , ) ( , 1)... ( , 1) ( , )kk kk kk kk k(k +1)i j i j i j - i j - L+ i j U u u u u                

Tˆ ˆ,... ( , +1)... ( , +1)]k(k +1) k(k +K -1)i j - L i j - L u u                       

(9)

where in this equation

ˆ ( ) = [ ( , ) ( , -1) (10) kk i, j u i - k j u i - k juT

2( , - 2)..... ( , +1)] u i - k j u i - k j - N       

By using this newly defined matrix, the following procedure can be used to find the positions of 1 blocks:

Computing Tˆ ˆTr ( ( , ) ( , ))kk kki j i j U U for

10 ( -1)k N .

The indices of k are correspondent to S largest

values of Tˆ ˆTr ( ( , ) ( , ))kk kki j i j U U .

B. 2D-PRA

This algorithm is one of those algorithms which can reduce the computational complexity, significantly. In this algorithm, the weight matrix is updated only once every ( )KL iterations.

Therefore, the computational cost of 2D-PRA is less than 2D-APA. The update equation for this algorithm can be expressed as

T -1

( , ) = ( , ) +μ ( , ) (11)

( ( , ) ( , ) + δ ) ( , )I

k +KL k k

k k k

i j i j i j

i j i j i j

w w U

U U e

Simulation results show that the performance of this algorithm is close to conventional 2D-APA adaptive filter.

C. 2D-SPR-APA

In some of two dimensional applications, it is necessary to reduce the computational load, as much as possible. To address this issue, we combined our two proposed algorithms, and established another new filtering method, namely two dimensional selective partial rank affine projection algorithm (2D-SPR-APA) with very low computational cost. The weight update equation for this algorithm can be obtained as

T -1

( , ) = ( , ) + ( , ) (12)

( ( , ) ( , ) + δ ) ( , )

I

w w A U

U A U e

k +KL k k k

k k k k

i j i j i j

i j i j i j

   

The computational complexity of this algorithm is less than 2D-SPU-APA and 2D-PRA, considerably. Actually, we have reduced the computational complexity in two different ways. First, we partition the weight matrix into blocks and update only one or several of these blocks at each iteration. Second we only update the weight coefficients of the filter once at each ( )KL iterations. Simulation results indicate that the

performance of this algorithm is close to 2D-SPU-APA, which means that, good performance and low computational complexity can be achieved simultaneously.

III. COMPUTATIONAL COMPLEXITY

TABLE Ι presents the computational complexity of those 2D adaptive algorithms which are introduced in this paper. As expected, the computational complexities of 2D-SPU-APA and 2D-RPA are less than conventional 2D-APA. Also, the computational complexity of 2D-SPR-APA adaptive filter is less than its counterparts.

International Conference on Electronics, Biomedical Engineering and its Applications (ICEBEA'2012) Jan. 7-8, 2012 Dubai

245

TABLE Ι: THE COMPUTATIONAL COMPLEXITY OF 2D-APA, 2D-SPU-APA, 2D-PRA AND 2D-SPR-APA.

 

. IV. SIMULATION RESULTS

In this section, we apply our proposed 2D adaptive filters to two different applications, which are 2D system identification and noise cancellation.

A. 2D System Identification

Figure. 1 shows the procedure of 2D system identification in a block diagram form. In the problem of system identification, the main goal is to tune the coefficient of the filter to generate outputs as similar as possible to those which are produced by the unknown system. In this section, we want to demonstrate the applicability and efficiency of our proposed 2D adaptive filters in the application of system identification.

Fig. 1: 2D system identification setup

In this simulation, the size of the 2D filter is

1 2 5N N .  ( , )v i j is a Gaussian signal with zero mean

and unit variance, while ˆ( , )v i j is the additive white noise

with zero mean and variance 2 0.01v

 and 2 0.1

v . In

addition, the 2D unknown system used in these simulations is created at random which has the same size as adaptive filter. Also ( , )u i j is the output of the low-pass filter which

can be expressed versus its input as  

( , ) + ( , -1) + ( -1, ) + ( , - 2)u i j au i j au i j bu i j

+ ( - 2, ) + ( -1, -1) + ( - 2, - 2) (13)2 2bu i j a u i j b u i j

+ ( -1, - 2) + ( - 2, -1) = ( , )abu i j abu i j v i j

Based on Fig.1, desired and error signal at any specific point can be written as follow

0 ˆ( , ) ( , ) ( , )d i j y i j v i j                                                  (14)

( , ) ( , ) ( , )e i j d i j y i j                                                   (15)

 

 

 

 

 

To compare the performance of the presented algorithms, with each other and with previously developed methods, we plot the MSD curves for each of these filtering methods. Definition of this criterion is given here for convenience.

where w specifies the weight matrix of unknown system. In all of the simulations, one of algorithms has been selected as a reference to have better comparison. Fig. 2 shows the performance of the 2D-SPU-APA with different number of blocks. Fig. 2(a) shows the case in which a=b=0

(therefore input of the system is a white noise), 2 0.01v

,

= = 2K L , and 0.01 for reference algorithm, 2D-APA.

While Fig. 2(b) shows the case in which a=-0.2, b=0.6

(therefore input of the system is a colored noise), 2 0.01v

, = = 2K L and 0.01  for 2D-APA.  

 

 

.

Fig. 2: The performance of 2D-SPU-APA with different number of blocks in update equation for (a) white noise (b) colored noise.

Algorithm Multiplications Divisions Comparisons Additional

multiplications

2D-APA 1 1

2 32 2( +1)( ) + 2( ) + ( )NKL N KL N N KL

----- -----

-----

2D-SPU-APA 2 32 2( +1)( ) + 2( ) + ( )KL SN KL SN KL ----- 1 2 1S+ O (N )N log 1

2D-PRA 1 1

22 2( +1)( ) + 2 + ( )NKL N N N KL ----- ----- -----

2D-SPR-APA 22 2( +1)( ) + 2 + ( )KL SN SN KL ----- ----- -----

ˆ( , )v i j

Unknown System

2D Adaptive Filter

0 ( , )y i j

( , )y i j

( , )d i j

( , )e i j

( , )u i j( , )v i j Lowpass Filter

(a)

2

10 2MSD 10 log (16)

w w

w

k

(b)

0 1 2 3 4 5 6 7

x 104

-60

-50

-40

-30

-20

-10

0

Iteration number

MS

D (d

B)

(a) (b) (c) (d) (e)

(a)2D-SPU-APA (S=1)(b)2D-SPU-APA (S=2)(c)2D-SPU-APA (S=3)(d)2D-SPU-APA (S=4)(e)2D-APA

Input: white Gaussian noise

0 1 2 3 4 5 6 7

x 104

-60

-50

-40

-30

-20

-10

0

Iteration Number

MS

D (d

B)

(a) (b) (c) (d) (e)

(a)2D-SPU-APA (S=1)(b)2D-SPU-APA (S=2)(c)2D-SPU-APA (S=3)(d)2D-SPU-APA (S=4)(e)2D-APA

Input: colored Gaussian noise

International Conference on Electronics, Biomedical Engineering and its Applications (ICEBEA'2012) Jan. 7-8, 2012 Dubai

246

Fig. 3: The performance of 2D-PRA algorithm for (a) white noise (b) colored noise.

Fig. 4: The performance of 2D-SPR-APA algorithm for (a) white noise (b) colored noise

Clearly, with the reduction of number of blocks, the convergence speed is decreased. In addition, the results show that in some cases, the difference between the performances of two algorithms, for example 2D-APA and 2D-SPU-APA ( 4)S is negligible which proves that, our

proposed algorithm, in spite of its low computational complexity, has a good performance in this application.

Fig. 3 shows the performance of 2D-PRA. In Fig. 3(a),

we have compared the performances of 2D-APA and 2D-PRA in two cases: K = L = 2 and K = L = 3 Also, in

this Fig, we considered a=b=0, 2 0.1v and 0.01   for 

reference algorithm, 2D-APA with K = L = 3 . Using 2D-PRA with K = L = 2 and K = L = 3 reduce the number of weight updating from 40000 to 10000 and 4444, respectively. In Fig. 3(b) we compared the performances of 2D-APA and 2D-PRA with 2 1K = ,L = and

1 3K = ,L = . In this Fig, we set a=-0.2, b=0.6, 

2 0.1v

and 0.01   for 2D-APA with 1 3K = ,L = . The number

of weight updating in 2D-PRA  for cases of 2 1K = ,L =  and  1 3K = ,L = was decreased from 40000 to 20000 and 13333, respectively. It is clear that even by this small number of weight updating, our proposed algorithm has demonstrated good performance in the application of 2D system identification. Furthermore, by considering smaller values for K and L ,  the performance of this algorithm would become very close to the conventional 2D-APA.  Fig. 4 shows the performance of 2D-SPR-APA compared with 2D-SPU-APA. In Fig. 4(a) we have chosen a=b=0, 

2K = L = , 2 0.1v , and = 0.02 for the case of

reference algorithm, 2D-SPU-APA ( = 3)S . In Fig 4(b) we

have considered a=-0.2, b=0.6,  2K = L = , 2 0.1v and

= 0.02 for the case of 2D-SPU-APA ( = 4)S . From these

simulation results, it is evident that regardless of the type of noise and the number of blocks, the performance of 2D-SPR-APA is comparable with the performance of the 2D-PRA.

B. Noise Cancellation

Now, in this section, we want to investigate the performance of our proposed 2D filtering algorithms in the application of image processing.

To eliminate noise in digital images, 2D adaptive filters can be used as a noise canceller in a configuration similar to what is depicted in Fig. 5.

The adaptive noise canceller tries to omit noise from the input noisy signal. In this case and after the convergence of filter coefficients, output error signal, i.e. ( )e i, j defined

below will be the estimation of desired signal.

(a)

(b)

(a)

(b)

0 0.5 1 1.5 2 2.5 3 3.5 4

x 104

-50

-40

-30

-20

-10

0

Iteration Number

MS

D (d

B)

(a)2D-APA (K=L=3)(b)2D-APA (K=L=2)(c)2D-PRA (K=L=3)(d)2D-PRA (K=L=2)

(b) (c) (d)(a)

Input: white Gaussian noise

0 0.5 1 1.5 2 2.5 3 3.5 4

x 104

-40

-30

-20

-10

0

Iteration Number

MS

D (d

B)

(a)2D-APA (K=1,L=3)(b)2D-APA (K=2,L=1)(c)2D-PRA (K=2,L=1)(d)2D-PRA (K=1,L=3)

Input: colored Gaussian noise

(a)(b) (c)(d)

0 0.5 1 1.5 2 2.5 3 3.5 4

x 104

-50

-40

-30

-20

-10

0

Iteration Number

MS

D (d

B)

(a)2D-SPU-APA (S=3)(b)2D-SPR-APA (S=3)(c)2D-SPU-APA (S=1)(d)2D-SPR-APA (S=1)

Input: white Gaussian noise (d) (c) (b) (a)

0 0.5 1 1.5 2 2.5 3 3.5 4

x 104

-50

-40

-30

-20

-10

0

Iteration Number

MS

D (d

B)

(a)2D-SPU-APA (S=4)(b)2D-SPU-APA (S=2)(c)2D-SPR-APA (S=4)(d)2D-SPR-APA (S=2)

Input: colored Gaussian noise (a)(b) (c) (d)

International Conference on Electronics, Biomedical Engineering and its Applications (ICEBEA'2012) Jan. 7-8, 2012 Dubai

247

Fig. 5. 2D Adaptive noise cancellation setup

The adaptive noise canceller tries to omit noise from the input noisy signal. In this case and after the convergence of filter coefficients, output error signal, i.e. ( )e i, j defined

below will be the estimation of desired signal.

2 ˆ( ) ( ) ( ) ( ) (17)e i, j = d i, j -v i, j x i, j

In this simulation  ( , )v i j is considered to be a white

Gaussian noise with zero mean and unit variance. The utilized low-pass filter which create the input color noise of the 2D adaptive filter can be expressed as

-1 -2 -3 -4 -51 1 1 1 1 1( ) 1 0.7 0.5 0.05 0.0056 0.0004b z = - z + z - z + z - z  

-1 -2 -3 -4 -52 2 2 2 2 2( ) 1 0.7 0.5 0.045 0.0046 0.0003b z = - z + z - z + z - z

1 2 1 2( ) ( ) ( )B z ,z = b z b z  

1 2 1( ) ( ) ( )B z ,z v i, j = v i, j                                                    (18)

In the following conducted experiments, to compare the performance of different 2D adaptive filters including our proposed methods, the PSNR criterion is calculated for the output image. Definition of this criterion is presented here for convenience

where I and J are the original and noisy images,

respectively, and F

Trace(( ) ( ))TI - J I - J I - J is the

frobenius norm. Fig. 6(a) displays the input noisy image. The input PSNR

set in 0 dB. TABLE ΙΙ reveals the result of applying 2D filtering algorithms to eliminate noise from image. In this table, our proposed algorithms with low computational complexity feature (i.e.2D-PRA, 2D-SPU-APA and 2D-SPR-APA) and 2D-APA are compared with each other. Figs. 6(b) to 6(i) show the final restored images by using these algorithms.

These presented results show that our novel filtering algorithms are successful in removing noise from images.

V. CONCLUSION In this paper we presented several 2D adaptive algorithms with low computational complexity, namely 2D-SPU-APA, 2D-PRA and 2D-SPR-APA. The performances of each of these algorithms are demonstrated in two of the

most important applications of 2D adaptive filters. In the application of 2D system identification, these algorithms have acceptable convergence speed and low steady state error. In addition, in the application of noise cancellation from images, the output PSNR and restored images confirm that these 2D filters act perfectly in removing noise from digital images.

(a)

Low pass

2D Adaptive Filter

( )v i, j 1 ( )v i, j 2 ( )v i, j

ˆ( ) ( ) ( )d i, j = x i, j + v i, j

( )e i, j

2

F10

1 2

PSNR = -10log (19)

I - J

M M

(b)

(c)

(d)

International Conference on Electronics, Biomedical Engineering and its Applications (ICEBEA'2012) Jan. 7-8, 2012 Dubai

248

TABLE ΙΙ: COMPARISON OF THE PERFORMANCE OF 2D ALGORITHMS

REFERENCES [1] Mohiy M. Hadhoud, David W. Thomas, “The tow-dimensional

adaptive LMS algorithm,” IEEE Trans. Circuits Systems, vol. 35, no. 5, pp. 485-494, 1988.

Fig. 6: (a) Noisy image. Restored images by (b) 2D-APA (K=L=2) (c) 2D-APA (K=L=3) (d) 2D-PRA (K=L=2) (e) 2D-PRA (K=L=3) (f) 2D-SPU-APA ( 3)S (g) 2D-SPU-APA ( 1)S

(h) 2D-SPR-APA ( 3)S (i) 2D-SPR-APA ( 1)S . [2] Mitsuji Muneyasu. Takao Hinamoto, “A realization of TD adaptive

filters using affine projection algorithm,” J.Franklin. vol. 335B, no. 7, pp. 1185-1193, 1998.

[3] George-Othon Glentis, “An efficient affine projection algorithm for TD FIR adaptive filtering and linear prediction,” Signal Processing. vol. 86, pp. 98-116, 2005.

[4] Mitsuji Muneyasu, Eiji Uemoto, and Takao Hinamoto, “A novel 2-d adaptive filter based on the 1-d RLS algorithm,” in IEEE International Symposium on Circuits and Systems, Hong Kong, pp. 2317-2320, June. 1997.

[5] A. M. Sequeria and C. W. Therrien, “Anew 2-d fast RLS algorithm,” in IEEE Proc. Int. Conf. On Acoust. Speech and Signal Proc., Albuquerque, New Mexico, pp. 1401-1404, Apr. 1990.

[6] S. C. Douglas, “Analysis and implementation of the max-NLMS adaptive filter,” in Proc. 29th Asilomar Conf. on Signals, Systems, and Computers, Pacific Grove, CA, pp. 659–663, Oct. 1995.

[7] K. Dogancay and O. Tanrıkulu, “Adaptive filtering algorithms with selective partial updates,” IEEE Trans. Circuits, Syst. II: Analog and Digital Signal Processing, vol. 48, no. 8, pp. 762–769, Aug. 2001.

[8] K.Y. Hwang, W.J. Song, “ An affine projection adaptive filtering algorithm with selective regressor,” IEEE Trans. Circuits Syst. Express Briefs 54, pp. 43-46, January. 2007.

[9] M. S. E. Abadi and J. H. Husøy, “Selective partial update and set- membership subbsnd adaptive filters,” Signal Process, vol. 88, pp.

2463-2471. 2008. [10] Sayed, A. H. Fundamentals of Adaptive Filtering. Wiley, 2003.

Algorithm K

L PSNR(out)

2D-APA 0.02 2 2 20.18 2D-APA 0.02 3 3 20.87 2D-PRA 0.02 2 2 19.41 2D-PRA 0.02 3 3 18.57

2D-SPU-APA ( = 3)S 0.01 2 2 19.89

2D-SPU-APA ( = 1)S 0.01 2 2 18.57

2D-SPR-APA ( = 3)S 0.01 2 2 19.31

2D-SPR-APA ( = 1)S 0.01 2 2 18.01

(f)

(i)

(e)

(g)

(h)

International Conference on Electronics, Biomedical Engineering and its Applications (ICEBEA'2012) Jan. 7-8, 2012 Dubai

249