channel estimation using svr - report (1)

Chanel Estimation Using SVR

Dept of ECE, PESIT, Bangalore

ABSTRACT

This project investigates the prospect of using support vector machine

(regression) as an algorithm for channel estimation. The Channel Estimation is a

fundamental component in a mobile OFDM software radio receiver. The channel

estimation techniques for OFDM systems based on pilot arrangement are

investigated. The channel estimation based on comb type pilot arrangement is

studied through different algorithms for both estimating channel at pilot

frequencies and interpolating the channel. The estimation of channel at pilot

frequencies is based on LS and LMS while the channel interpolation is done using

linear interpolation, second order interpolation, low-pass interpolation, spline cubic

interpolation, and time domain interpolation. Furthermore, the channel estimation

based on block type pilot arrangement is performed by sending pilots at every sub-

channel and using this estimation for a specific number of following symbols. We

have also implemented. We have compared the performance of BPSK by

measuring bit error rate by altering no. of pilots based on multi-path frequency

selective and AWGN channels as channel models.



Chapter 1

INTRODUCTION



1.1 SUBSYSTEMS IN DIGITAL COMMUNICATION SYSTEMS

Source Coding: Entropy of a source is the measure of information. Basically source codes

try to reduce the redundancy present in the source, and represent the source with fewer bits

that carry more information. Data compression which explicitly tries to minimize the average

length of messages according to a particular assumed probability model is called entropy

encoding. Various techniques used by source coding schemes try to achieve the limit of

Entropy of the source.

Channel Coding: The aim of channel coding theory is to find codes which transmit quickly,

contain many valid code words and can correct or at least detect many errors. While not

mutually exclusive, performance in these areas is a trade off. So, different codes are optimal

for different applications. The needed properties of this code mainly depend on the

probability of errors happening during transmission. In a typical CD, the impairment is

mainly dust or scratches. Thus codes are used in an interleaved manner. The data is spread

out over the disk. Although not a very good code, a simple repeat code can serve as an

understandable example. Suppose we take a block of data bits (representing sound) and send

it three times. At the receiver we will examine the three repetitions bit by bit and take a

majority vote. The twist on this is that we don't merely send the bits in order. We interleave

them. The block of data bits is first divided into 4 smaller blocks. Then we cycle through the

block and send one bit from the first, then the second, etc. This is done three times to spread

the data out over the surface of the disk. In the context of the simple repeat code, this may

not appear effective. However, there are more powerful codes known which are very

effective at correcting the "burst" error of a scratch or a dust spot when this interleaving

technique is used.

Modulation and Demodulation: In digital modulation, an analog carrier signal is modulated

by a discrete signal. Digital modulation methods can be considered as digital-to-analog



conversion, and the corresponding demodulation or detection as analog-to-digital

conversion.

Fig 1.1 .Basic block diagram of digital communication

1.2 AWGN (Additive white Gaussian noise) channel.

We make the standard assumption that w(t) is a zero mean additive white gaussian

noise(AWGN) with power spectral density No/2

E[w(0)w(t)] =

The output signal received through AWGN channel

y (n) = x(n) + w(n)

The capacity for the AWGN channel is given by

C = B log (1 + SNR)

Source coding Channel coding Modulator

Demodulator Channel decoding Source decoding

Message

symbols

Received Message

symbols



AWGN channel model is best suited for deep space communication and satellite communication.

But this model is not suitable for terrestrial communication where the signal is obstructed by

obstacles.

w(n)

x(n) Σ y(n)

1.3 FADING CHANNELS

In wireless communications, fading is deviation of the attenuation that a carrier-

modulated telecommunication signal experiences over certain propagation media. The fading

may vary with time, geographical position and/or radio frequency, and is often modelled as

a random process. A fading channel is a communication channel that experiences fading. In

wireless systems, fading may either be due to multipath propagation, referred to as multipath

induced fading, or due to shadowing from obstacles affecting the wave propagation, sometimes

referred to as shadow fading.

w(n)

x(n) H Σ y(n) (HHH)

-1H

H

As the carrier frequency of a signal is varied, the magnitude of the change in amplitude will vary.

The coherence bandwidth measures the separation in frequency after which two signals will

experience uncorrelated fading.

There are mainly two types of fading channels

1. Flat fading channel

2. Frequency selective fading channel

In flat fading, the coherence bandwidth of the channel is larger than the bandwidth of the

signal. Therefore, all frequency components of the signal will experience the same magnitude

of fading.The output signal for this type of channel is given by

Fig 1.2. AWGN Channel Model

Fig 1.3. Fading Channel Model



y = hx +w

here convolution becomes element wise multiplication,

In frequency-selective fading, the coherence bandwidth of the channel is smaller than the

bandwidth of the signal. Different frequency components of the signal therefore experience

decorrelated fading.

y(n)= ΣN x(l) h(n-l) + w(n)

1.4 RAYLEIGH FADING

Rayleigh fading is a statistical model for the effect of a propagation environment on

a radio signal, such as that used by wireless devices.

Rayleigh fading models assume that the magnitude of a signal that has passed through

such a transmission medium (also called a communications channel) will vary randomly, or fade,

according to a Rayleigh distribution — the radial component of the sum of two

uncorrelated Gaussian random variables.

X = Xr + jXi

Rayleigh fading is viewed as a reasonable model for tropospheric and ionospheric signal

propagation as well as the effect of heavily built-up urban environments on radio

signals. Rayleigh fading is most applicable when there is no dominant propagation along a line

of sight between the transmitter and receiver. If there is a dominant line of sight, Rician

fading may be more applicable.



Chapter 2

ORTHOGONAL FREQUENY DIVISION

MULTIPLEXING



2.1 OFDM (Orthogonal Frequency Division Multiplexing)

OFDM is a combination of modulation and multiplexing. Multiplexing generally refers to

independent signals those produced by different sources. so it is a question of how to share the

spectrum with these users. In OFDM multiplexing is applied to independent signals but these

independent signals are a subset of one main signal. In OFDM the signal itself is first split into

independent channels, modulated by data and then re-multiplexed to create the OFDM carrier.

OFDM is a special case of frequency division multiplexing.

In OFDM we have N carriers, N can be anywhere from 16 to 1024 in present technology

and depends on the environment in which the system will be used. Let‘s examine the following

bit sequence we wish to transmit and show the development of the OFDM signal using 4 sub-

carriers. The signal has a symbol rate of 1 and sampling frequency is 1sample per symbol, so

each transition is a bit.

message signal

A bit stream is modulated using a 4 carrier OFDM. First few bits are 1, 1, -1, -1, 1, 1, 1, -

1, 1, -1, -1, -1, -1, 1, -1, -1, -1, 1,…Let‘s now write these bits in rows of fours, since this

demonstration will use only four subcarriers. We have effectively done a serial to parallel

conversion. Table I – Serial to parallel conversion of data bits.

c1 c2 c3 c4

1 1 -1 -1

1 1 1 -1

1 -1 -1 -1

-1 1 -1 -1

-1 1 1 -1

-1 -1 1 1



Table- 1

Each column represents the bits that will be carried by one sub-carrier. Let‘s start with the first

carrier, c1. What should be its frequency? From the Nyquist sampling Theorem, we know that

smallest frequency that can convey information has to be twice the information rate. In this case,

the information rate per carrier will be 1/4 or 1 symbol per second total for all 4 carriers. So the

smallest frequency that can carry a bit rate of 1/4 is 1/2 Hz. But we picked 1 Hz for convenience

Had I picked 1/2 Hz as my starting frequency, then my harmonics would have been 1, 3/2 and 2

Hz. I could have chosen 7/8 Hz to start with and in which the harmonics would be 7/4, 7/2, 21/2

Hz.

We pick BPSK as our modulation scheme for this example. (For QPSK, just imagine the

same thing going on in the Q channel, and then double the bit rate while keeping the symbol rate

the same). Note that I can pick any other modulation method, QPSK, 8PSK 32-QAM or

whatever No limit here on what modulation to use. I can even use TCM which provides coding

in addition to modulation.

Carrier 1 - We need to transmit 1, 1, 1 -1, -1, -1 which I show below superimposed on the BPSK

carrier of frequency 1 Hz. First three bits are 1 and last three -1. If I had shown the Q channel of

this carrier (which would be a cosine) then this would be a QPSK modulation.

Sub-carrier 1 and the bits it is modulating (the first column of Table 1)

Carrier 2 - The next carrier is of frequency 2 Hz. It is the next orthogonal/harmonic to frequency

of the first carrier of 1 Hz. Now take the bits in the second column, marked c2, 1, 1, -1, 1, 1, -1

and modulate this carrier with these bits as shown in fig.



Sub-carrier 2 and the bits that it is modulating (the 2nd column of Table I)

Carrier 3 – Carrier 3 frequency is equal to 3 Hz and fourth carrier has a frequency of 4 Hz. The

third carrier is modulated with -1, 1, 1, -1, -1, 1 and the fourth with -1, -1, -1, -1, -1, -1, 1 from

Table I.

Sub-carrier 3 and 4 and the bits that they modulating( the 3

rd and 4

th column of table1)

Now we have modulated all the bits using four independent carriers of orthogonal frequencies 1

to 4 Hz. What we have done is taken the bit stream, distributed the bits, one bit at a time to the

four sub-carriers as shown in figure below.



Now add all four of these modulated carriers to create the OFDM signal, often produced by a

block called the IFFT.

X

X

X

Functional diagram of an OFDM signal creation. The outlined part is often called an IFFT block

The generated OFDM signal. Note how much it varies compared to the underlying constant

amplitude sub-carriers.

In short-hand, we can write the process above as

∑ (t) sin(2Πnt)

Fig 1.4. IFFT block diagram

Bit Stream

Parallel

to serial

converter



It is basically an equation of an Inverse FFT.

Using Inverse FFT to create the OFDM symbol.The equation is essentially an inverse FFT. The

IFFT block in the OFDM chain confuses people. So let‘s examine briefly what an FFT/IFFT

does.

Time domain view

Forward FFT takes a random signal, multiplies it successively by complex exponentials

over the range of frequencies, sums each product and plots the results as a coefficient of that

frequency The coefficients are called a spectrum and represent ―how much‖ of that frequency is

present in the input signal. The results of the FFT in common understanding is a frequency

domain signal.

Frequency Domain View



We can write FFT in sinusoids as

∑ (n) sin(

)+ ∑

(n) cos(

)

The inverse FFT takes this spectrum and converts the whole thing back to time domain signal by

again successively multiplying it by a range of sinusoids.

The equation for IFFT is

∑ (k) sin(

)- ∑

(k) cos(

)

The difference between Eq. 5 and 6 is the type of coefficients the sinusoids are taking, and the

minus sign, and that‘s all. The coefficients by convention are defined as time domain samples

x(k) for the FFT and X(n) frequency bin values for the IFFT.

The two processes are a linear pair. Using both in sequence will give the original result back.

The block diagram for OFDM scheme is as shown below

Due to the addition of cyclic prefix inter symbol interference gets avoided. Also linear

convolution of the transmitted signal with the channel response becomes circular

IFFT Addition of

cyclic prefix

DAC

ADC Removal of

cyclic prefix

FFT

Fig 1.5. OFDM Block diagram



convolution.Due to this each subcarrier experiences flat fading, even in a frequency selective

channel. This allows the use of simple detectors at the receiver.

2.2 LINEAR ALGEBRA APPROACH

In linear algebra, a square matrix A is called diagonalizable if it is similar to a diagonal

matrix, i.e., if there exists an invertible matrix P such that P −1

AP is a diagonal matrix. If V is a

finite-dimensional vector space, then a linear map T : V → V is called diagonalizable if there

exists a basis of V with respect to T which is represented by a diagonal matrix. Diagonalisation is

the process of finding a corresponding diagonal matrix for a diagonalizable matrix or linear map.

A square matrix which is not diagonalizable is called defective.

Diagonalizable matrices and maps are of interest because diagonal matrices are especially

easy to handle: their eigenvalues and eigenvectors are known and one can raise a diagonal matrix

to a power by simply raising the diagonal entries to that same power. Geometrically, a

diagonalizable matrix is an inhomogeneous dilation (or anisotropic scaling) – it scales the space,

as does a homogeneous dilation, but by a different factor in each direction, determined by the

scale factors on each axis (diagonal entries).

The fundamental fact about diagonalizable maps and matrices is expressed by the following:

An n-by-n matrix A over the field F is diagonalizable if and only if the sum of

the dimensions of its eigen spaces is equal to n, which is the case if and only if there exists

a basis of Fn

consisting of eigenvectors of A. If such a basis has been found, one can form the

matrix P having these basis vectors as columns, and P−1

AP will be a diagonal matrix. The

diagonal entries of this matrix are the eigenvalues of A.

A linear map T: V → V is diagonalizable if and only if the sum of the dimensions of its

eigen-spaces is equal to dim(V), which is the case if and only if there exists a basis

of V consisting of eigenvectors of T. With respect to such a basis, T will be represented by a

diagonal matrix. The diagonal entries of this matrix are the eigenvalues of T.

Another characterization: A matrix or linear map is diagonalizable over the field F if and only if

its minimal polynomial is a product of distinct linear factors over F. (Put in another way, a

matrix is diagonalizable if and only if all of its elementary divisors are linear.)



The following sufficient (but not necessary) condition is often useful.

An n-by-n matrix A is diagonalizable over the field F if it has n distinct eigenvalues in F, i.e.

if its characteristic polynomial has n distinct roots in F; however, the converse may be false.

If a matrix A can be diagonalized, that is,

then:

Writing P as a block matrix of its column vectors

Consider the Circulant Matrix:

[

]

this can be split up into sum of the matrices as shown below:

h0* [

] + h1*[

] + h2*[

] + h3*[

]



From the procedure stated above if we find the eigen vectors of these matrices we get the

transformation matrix as

P = [

]

If we look into the DFT matrix for a 4 point signal we get it as follows:

We can observe that it is just a scaled version of the transformation matrix P. Hence we conclude

that DFT matrix will digonalise the circulant matrix. This implies that Decoding tha symbol

patterns is just point by point division instead of matrix inversion. Hence decoder is quite simple

and feasible if we know the Channel response.

2.3 ADVANTAGES OF OFDM

OFDM signaling offers several advantages, which are listed below

1. In the deep channel fading , the received signal in single carrier is erroneous. But in the

multicarrier mode like OFDM, each signal is transmitted in different carrier and the

probability of error is very low.

2. Due to the addition of cyclic prefix the ISI is avoided if the system is perfectly

synchronized. The linear convolution becomes circular convolution. Due to this, even in

frequency selective fading, subcarriers experience flat fading which allows use of simple

detectors at the receiver.

3. Since the subcarriers are orthogonal spectrum can overlap. This makes OFDM scheme as

bandwidth efficient.

4. OFDM allows multi-user in the system where different carriers are allotted to different

users.

5. Implementation cost is low because of the use of IFFT and FFT processing.



Chapter 3

Support Vector Regression



3.1 SUPPPORT VECTOR MACHINES

Suppose we are given training data {(x1, y1),……. (xi, yi)} ⊂ X × R, where X denotes the

space of the input patterns (e.g. X = Rd). These might be, for instance, exchange rates for some

currency measured at subsequent days together with corresponding econometric indicators. In ε-

SV regression [Vapnik, 1995], our goal is to find a function f(x) that has at most ε deviation from

the actually obtained targets yi for all the training data, and at the same time is as flat as possible.

In other words, we do not care about errors as long as they are less than ε, but will not accept any

deviation larger than this. This may be important if you want to be sure not to lose more

than ε money when dealing with exchange rates, for instance.For pedagogical reasons, we begin

by describing the case of linear functions f, taking the form

f(x) = (w, x) + b with w ∈ X, b ∈ R …………….(1)

Where ‗.’ denotes the dot product in X. Flatness in the case of (1) means that one seeks a small

w. One way to ensure this is to minimize the norm,3 i.e. ||w||2 = (w,w). We can write

this problem as a convex optimization problem:

The tacit assumption in (2) was that such a function f actually exists that approximates all

pairs (xi, yi) with ε precision, or in other words, that the convex optimization problem is feasible.

Sometimes, however, this may not be the case, or we also may want to allow for some errors.

Analogously to the ―soft margin‖ loss function [Bennett and Mangasarian, 1992] which was

adapted to SV machines by Cortes and Vapnik [1995], one can introduce slack variables ξi, ξi to

cope with otherwise infeasible constraints of the optimization problem (2). Hence we arrive at the

formulation stated in [Vapnik, 1995].



The constant C > 0 determines the trade-off between the flatness of f and the amount up to which

deviations larger than ε are tolerated. This corresponds to dealing with a so called ε–insensitive

loss function described by

Figure depicts the situation graphically. Only the points outside the shaded region contribute to

the cost insofar, as the deviations are penalized in a linear fashion. It turns out that in most cases

the optimization problem (3) can be solved more easily in its dual formulation. Moreover the

dual formulation provides the key for extending SV machine to nonlinear functions. Hence we

will use a standard dualization method utilizing Lagrange multipliers, as

described in e.g. [Fletcher, 1989].

3.2 SUPPORT VECTOR REGRESSION

In statistics, linear regression is an approach to modeling the relationship between a

scalar dependent variable y and one or more explanatory variables denoted X. The case of one

explanatory variable is called simple regression. More than one explanatory variable is multiple

regression. (This in turn should be distinguished from multivariate linear regression, where

multiple correlated dependent variables are predicted rather than a single scalar variable.)

In linear regression, data are modeled using linear predictor functions, and unknown

model parameters are estimated from the data. Such models are called linear models. Most



commonly, linear regression refers to a model in which the conditional mean of y given the value

of X is an affine function of X. Less commonly, linear regression could refer to a model in which

the median, or some other quantile of the conditional distribution of y given X is expressed as a

linear function of X. Like all forms of regression analysis, linear regression focuses on

the conditional probability distribution of y given X, rather than on the joint probability

distribution of y and X, which is the domain of multivariate analysis.

Linear regression was the first type of regression analysis to be studied rigorously, and to be used

extensively in practical applications. This is because models which depend linearly on their

unknown parameters are easier to fit than models which are non-linearly related to their

parameters and because the statistical properties of the resulting estimators are easier to

determine.

Linear regression has many practical uses. Most applications of linear regression fall into one of

the following two broad categories:

If the goal is prediction, or forecasting, linear regression can be used to fit a predictive model

to an observed data set of y and X values. After developing such a model, if an additional

value of X is then given without its accompanying value of y, the fitted model can be used to

make a prediction of the value of y.

Given a variable y and a number of variables X1,….. Xp that may be related to y, linear

regression analysis can be applied to quantify the strength of the relationship between y and

the Xj, to assess which Xj may have no relationship with y at all, and to identify which

subsets of the Xj contain redundant information about y.

Linear regression models are often fitted using the least squares approach, but they may also be

fitted in other ways, such as by minimizing the ―lack of fit‖ in some other norm (as with least

absolute deviations regression), or by minimizing a penalized version of the least squares loss

function as in ridge regression. Conversely, the least squares approach can be used to fit models

that are not linear models. Thus, while the terms "least squares" and "linear model" are closely

linked, they are not synonymous.



Here is an example for linear regression.

Non-linear Regression:

In statistics, nonlinear regression is a form of regression analysis in which observational

data are modeled by a function which is a nonlinear combination of the model parameters and

depends on one or more independent variables. The data are fitted by a method of successive

approximations.

The data consist of error-free independent variables (explanatory variables), x, and their

associated observed dependent variables (response variables), y. Each y is modeled as a random

variable with a mean given by a nonlinear function f(x,β). Systematic error may be present but its

treatment is outside the scope of regression analysis. If the independent variables are not error-

free, this is an errors-in-variables model, also outside this scope.



Here is an example of Non-Linear Regression. All the curves drawn with colors are

approximations but they have different set of statistics such as mean square error etc.



CHAPTER 4

RESULTS



4.1 AWGN CHANNEL

The channel is assumed to be White, Additive and Gaussian in nature. The response for which is

obtained as shown in the figure. The Probability Distribution is as shown below.

The received signal,

when bit 1 is transmitted and

when bit 0 is transmitted.

The conditional probability distribution function (PDF) of for the two cases are:

.

Symbollically it can be depicted as shown below:



BER Performance for AWGN Channel

The Bit Error Rate Performance for the AWGN Channel can be obtained. From the Graph we

can see that:

1. As the SNR (Signal to Noise Ratio) increases the Bit Error Rate decreases.

2. The theoretical and Simulated Curves overlap on each other as expected.

4.2 RAYLEIGH (FLAT FADING) CHANNEL:

If we recall, in the post on BER computation in AWGN, the probability of error for

transmission of either +1 or -1 is computed by integrating the tail of the Gaussian probability

density function for a given value of bit energy to noise ratio . The bit error rate is,

.



However in the presence of channel , the effective bit energy to noise ratio is . So the

bit error probability for a given value of is,

,

where .

To find the error probability over all random values of , one must evaluate the

conditional probability density function over the probability density function of .

Probability density function of

From our discussion on chi-square random variable, we know that if is a Rayleigh distributed

random variable, then is chi-square distributed with two degrees of freedom. since is chi

square distributed, is also chi square distributed. The probability density function of is,

.

Error probability:

So the error probability is,

.

Somehow, this equation reduces to BER as:



BER Performance for Flat-Fading Channel:

The same results follow here as in the case of AWGN Channel. But we get different set

of curves as the variation in SNR.

4.3 BER PERFORMANCE FOR FREQUENCY SELECTIVE CHANNEL:

From the post on BER for BPSK in Rayleigh channel, the BER for BPSK in a Rayleigh

fading channel is defined as

We know that Fourier transform of a Gaussian random variable is still has a Gaussian

distribution. So, I am expecting that the frequency response of a complex Gaussian random

variable (a.k.a Rayleigh fading channel) will be still be independent complex Gaussian random

variable over all the frequencies.

Hence the graphs obtained for the performance is as shown below:



4.4 ESTIMATION OF FREQUENCY RESPONSE:

Using classical method of interpolation we tried to interpolate the response of the

Channel by using different number of pilot carriers. As we can see that the estimation improves

with the number of pilot carriers transmitted. We can see that there is a trade off between:

1. Band-Width

2. Accuracy of Estimation.

The figures below present the estimation with different number of pilot carriers namely

4,8,16. the interpolation methods used are Nearest Neighbor, Linear, Cubic and Spline.



4.5 REGRESSION USING SVM (LS-SVM):

Least Square SVM is chosen for the interpolation and training the machine. The standard

library designed by LSSVM [5] is used for all the simulations. The below graphs depict the

estimation obtained using different number of carrier transmissions.

Below shown are the graphs of real and imaginary parts of a frequency selective channel.



CHAPTER 5

CONCLUSION

The estimation of channel at pilot frequencies is based on LS and LMS

while the channel interpolation is done using linear interpolation, second order

interpolation, low-pass interpolation, spline cubic interpolation, and time domain

interpolation. Furthermore, the channel estimation based on block type pilot

arrangement is performed by sending pilots at every sub-channel and using this

estimation for a specific number of following symbols. We have also implemented.

We have compared the performance of BPSK by measuring bit error rate by

altering no. of pilots based on multi-path frequency selective and AWGN channels

as channel models.



CHAPTER 6

FUTURE WORK

These methods developed can be implemented on FPGA Kit. Analysis can

be done with respect various kernels for regression. Same kind of implementation

can be carried out of DPSK, QPSK, 16-QAM Systems.

These procedures are still under development and hence can be realised in

practice. Because of the efficiency and performance in different modulation

scheme.



BIBLIOGRAPHY

1) A TUTORIAL ON SUPPORT VECTOR REGRESSION BY ALEX J. SMOLA AND

BERNHARD SCH¨OLKOPF

2) BLOG ON DSP APPLICATIONS IN COMMUNIACTION SYSTEMS

http://www.dsplog.com/

3) SUPPORT VECTOR REGRESSION BY MAXWELLING, DEPARTMENT OF

COMPUTER SCIENCE, ORONTO UNIVERSITY CANADA

4) SVR LIBRARY DEVELOPED BY J.A.K. SUYKENS, T. VAN GESTEL, J. DE

BRABANTER, B. DE MOOR, J. VANDEWALLE, LEAST SQUARES SUPPORT

VECTOR MACHINES, WORLD SCIENTIFIC, SINGAPORE, 2002 (ISBN 981-238-

151-1)

5) COURSE ON MACHINE LEARNING OFFERED BY DR. ANDREW NG

ARTIFICIAL INTELLIGENCE LAB STANFORD UNIVERSITY CALIFORNIA.

www.ml-class.org

channel estimation using svr - report (1)

Documents