4818 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 7, NO. 12, DECEMBER 2008

Symbol and Bit Error Probabilities of Orthogonal Space-Time BlockCodes with Antenna Selection over Keyhole Fading Channels

Nghi H. Tran, Member, IEEE, Ha H. Nguyen, Senior Member, IEEE, and Tho Le-Ngoc, Fellow, IEEE

Abstract—The symbol error rate (SER) and the bit error rate(BER) of orthogonal space-time block codes (OSTBCs) withantenna selection over keyhole fading channels are examined.Considered are receive antenna selection, transmit antenna selec-tion, and joint antenna selection at both the transmitter and thereceiver. The exact SERs of OSTBCs for M -PSK and square M -QAM constellations are obtained using the technique of momentgenerating function (MGF). By applying the Bonferroni-typebounds, tight lower and upper bounds for both the SER and BERare provided in closed-form expressions with finite-range singleintegrals. The bounds can be applied to arbitrary constellationsand mappings. Numerical results show that the bounds can beused to provide practically the exact SER and BER over a widerange of the signal-to-noise ratio.

Index Terms—Antenna selection, keyhole fading, orthogonalspace-time block codes, Bonferroni-type bounds, symbol errorrate (SER), bit error rate (BER).

I. INTRODUCTION

SPACE-TIME code is a promising modulation techniqueto maximize the diversity gain over multiple-input and

multiple-output (MIMO) channels with multiple antennas atthe transmitter and the receiver [1], [2]. Due to its simpleimplementation while providing high diversity gains, orthog-onal space-time block codes (OSTBCs) [2], [3] have receiveda special attention, both in research community and industry.However, the deployment of OSTBCs in particular and MIMOsystems in general can be quite costly since the transmitterand/or the receiver require multiple radio frequency chainsand low-noise amplifiers.

To overcome this challenge, performing antenna selection isan attractive technique. This is because with antenna selectionthe system can still retain most of the advantages offered bymultiple antennas while the complexity of hardware imple-mentation is reduced [4]. Although transmit antenna selection

Manuscript received September 19, 2007; revised January 6, 2008 andApril 13, 2008; accepted April 21, 2008. The associate editor coordinatingthe review of this paper and approving it for publication was B. S. Rajan.This work was supported in part by Discovery Grants from the NaturalSciences and Engineering Research Council of Canada (NSERC). The firstauthor would also like to acknowledge the University of Saskatchewan’sGraduate Scholarship and the Fellowship received from TRlabs-Saskatoon.A part of this work was presented at the IEEE International Conference onCommunications (ICC), Beijing, China, May 19-23 2008.

N. H. Tran was with the Department of Electrical & Computer Engineering,University of Saskatchewan, Saskatoon, SK, Canada S7N 5A9. He is now withthe Department of Electrical & Computer Engineering, McGill University,Montreal, Quebec, Canada H3A 2A7. (e-mail: nghi.tran@mcgill.ca).

H. H. Nguyen is with the Department of Electrical & Computer Engineer-ing, University of Saskatchewan, Saskatoon, SK, Canada S7N 5A9. (e-mail:ha.nguyen@usask.ca.).

T. Le-Ngoc is with the Department of Electrical & Computer Engi-neering, McGill University, Montreal, Quebec, Canada H3A 2A7. (e-mail:tho@ece.mcgill.ca.).

Digital Object Identifier 10.1109/T-WC.2008.071041

and receive antenna selection has been well-investigated overthe conventional independent and identically distributed (i.i.d.)fading channel (see, for example, [5], [6] and many referencestherein), only a few efforts have been devoted to the caseof MIMO wireless channels with the existence of keyholefading, where scattering processes are present at both thetransmitter and the receiver [7], [8]. In particular, capacity anddiversity analysis for antenna selection over keyhole fadingwas investigated in [9], which shows that the performanceof systems using antenna selection can match to that of abaseline full-antenna system. Regarding OSTBCs, to the bestof our knowledge, only the approximation of the BER of asystem employing Alamouti code and BPSK modulation atthe transmitter and the selection of one best receive antennawas provided in [10].

This paper examines the error performance, both the SERand the BER, of OSTBCs over keyhole fading channels withjoint antenna selection at both the transmitter and the receiver,which includes receive antenna selection and transmit antennaselection as special cases. At first, by deriving the momentgenerating function (MGF) of an instantaneous signal-to-noiseratio (SNR) at the receiver, the exact SERs of OSTBCs forM -PSK and square M -QAM constellations are computed.Then the Bonferroni-type bounds are applied to obtain thetight lower and upper bounds on both the SER and BER.These bounds, which can be effectively computed with finite-range single integrals, are applicable to any constellations andmappings. It is shown that the bounds can be used to providepractically the exact SER and BER over a wide range of SNR.

It should be noted that this paper only considers the casethat the channel state information is perfectly estimated at thereceiver. The analysis with antenna selection in the presenceof channel estimation error, as similar to [11]–[13], would bean interesting topic for further studies.

II. SYSTEM MODEL AND THE TECHNIQUE OF MOMENT

GENERATING FUNCTION (MGF)

A. System Model

Consider a MIMO system with Lt transmit and Lr re-ceive antennas. The channel model for keyhole fading is [9]H = hLth

�Lr

where hLt and hLr are column vectors ofsize Lt and Lr, respectively, whose entries are circularlysymmetric complex Gaussian random variables with variance1/2 per dimension CN (0, 1). For receive antenna selection,the receiver will select Nr out of Lr antennas. If antennaselection is performed at the transmitter, it is assumed thatthe transmitter knows only the indices of Nt out of Lt

transmit antennas sent back from the receiver via a rate-limitedfeedback with error-free channel. Moreover, joint antenna

1536-1276/08$25.00 c© 2008 IEEE

IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 7, NO. 12, DECEMBER 2008 4819

selection with Nt transmit antennas and Nr receive antennacan also be implemented.

In a space-time block coded system, with Nt and Nr

selected antennas, each block of K · b information bits isfirst mapped to a sequence of K symbols {v1, v2, . . . , vK},where each vk can be one of M = 2b signal points {su}M

u=1

in a M -ary constellation Φ. The mapping rule ξ defines thecorrespondence between b bits and a signal point in Φ. Thesequence {v1, v2, . . . , vK} is then encoded into an N × Nt

space-time block code matrix G. Here, N is the number ofrows of the space-time code matrix and the code rate isR = K/N symbol per channel use. In the case of OSTBCs,the columns of G are orthogonal. The receive signals at the Nr

selected receive antennas over N time periods are collectivelygiven as:

R =√

ρ

NtGH + W . (1)

Here, H = hNth�Nr

with hNt and hNr correspond to Nt

and Nr selected components of hLt and hLr , respectively.The matrix W is a N × Nr matrix representing additivewhite Gaussian noise whose entries are also CN (0, 1). Thenormalization factor

√ρ

Ntin (1) ensures that the average SNR

at each receive antenna is ρ, independent of Nt.Due to the orthogonal structure of OSTBCs, it can be shown

that a MIMO fading channel can be equivalently representedby a single-input single-output (SISO) channel [14]. Let Y bethe squared Frobenius norm of H . The achievable SNR persymbol is given as:

ρs =ρR

Nt· Y =

ρR

Nt· U · V, (2)

where U = ||hNt ||2 and V = ||hNr ||2. From (2), it isobvious that the problem of joint transmit and receive an-tenna selection to maximize ρs can be decomposed into theproblems of transmit antenna selection and receive antennaselection independently. In particular, Nt and Nr transmit andreceive antennas should be selected to maximize U and V ,respectively.

B. MGF of Y and the Exact SER

In order to derive the SER and BER, this subsection firstevaluates the MGF of the random variable Y . Following theanalysis in [15], the probability density function (pdf) of therandom variable V is given as:

pV (v) =(

Lr

Nr

)[vNr−1 exp(−v)

Γ(Nr)+

Lr−Nr∑i=0

βiexp(−v)·⎛⎝exp

(− iv

Nr

)−

Nr−1∑j=0

ηi,jvj−1

⎞⎠

⎤⎦ , (3)

where

βi =

{(−1)i

(Lr−Nr

i

) (−Nr

i

)Nr−1i > 0

0 Otherwise(4)

ηi,j =

{1

Γ(j)

(−iNr

)j−1

j > 00 Otherwise

(5)

The pdf of U is also expressed in a similar form. Since Y =U · V and U and V are two independent random variables,one has:

pY (y) =∫ ∞

0

pV (v)pU

(y

v

) 1vdv. (6)

The moment generating function of Y with joint receive andtransmit antenna selection can then be computed as φ

(r,t)Y (s) =

E[exp(−sY )] =∫∞0 exp(−sy)pY (y)dy.

As shown in Appendix A, the pdf of Y , pY (y), andthe MGF of Y , φ

(r,t)Y (s), can be written in closed-form

expressions. In particular, from (28), it can be seen that theMGF of Y is expressed in a closed form as the sum of variousfunctions, each having the same form of f(s; A, B, C, D) =As−BU(B, C, D/s), where U(a, b, x) is the confluent hyper-geometric function of the second kind [16] and A, B, C, Dare some constants.

For example, in the case of receive antenna selection only,since Lt − Nt = 0 and the fact that β0 = 0 and ηi,0 = 0, theMGF of Y in (28) can be written in a more compact form as:

φ(r)Y (s) =

2

Γ(Lt)

(Lr

Nr

) [Γ(Lt)

2s−NrU(Nr, Nr − Lt + 1, 1/s)

+

Lr−Nr∑i=1

βi

[Γ(Lt)

2

(Nr + i

Nr

)Lt−1

s−LtU(

Lt, LtNr + i

Nrs

)

−Nr−1∑j=1

ηi,jΓ(j)Γ(Lt)

2s−jU (j, 1 + j − Lt, 1/s)

]].

(7)

With the closed-form expressions of the MGF of Y pre-sented in (28), the exact SER of OSTBCs for standard M -PSK and square M -QAM constellations can be computedstraightforwardly. More specifically, by averaging the condi-tional SERs for M -PSK and square M -QAM constellationsgiven in [17] over the pdf of the instantaneous SNR Y , oneobtains the following expressions for the general case of jointreceive and transmit antenna selection:

P MPSKsymbol(error) =

1π

∫ π−π/M

0

φ(r,t)Y

(ρRNt

gMPSK

sin2 θ

)dθ (8)

P MQAMsymbol (error) =

4q

π

∫ π/2

0

φ(r,t)Y

(ρRNt

gMQAM

sin2 θ

)dθ

− 4q2

π

∫ π/4

0

φ(r,t)Y

(ρRNt

gMQAM

sin2 θ

)dθ (9)

where φ(r,t)Y (·) is the MGF of Y , given in (28). It can be

seen that the single integrals in (8) and (9) are over finiteranges. Therefore, the exact SERs for M -PSK and square M -QAM can be computed easily. This fact holds for transmitantenna selection and receive antenna selection as well, sincethe MGFs of Y in all three cases have similar expressions.

III. TIGHT UPPER AND LOWER BOUNDS ON THE SER AND

BER BASED ON BONFERRONI-TYPE BOUNDS

Bonferroni-type bounds are the bounds on the probability ofa union. Let A1, A2, . . . , AX be X events defined on a given

4820 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 7, NO. 12, DECEMBER 2008

probability space. Then the probability of the union event is

PU = P

(X⋃

x=1

Ax

)=

X∑x=1

(−1)x+1Sx, (10)

where Sx =∑

P (Ax1 ∩· · ·∩Axk) and the sum is taken over

all 1 ≤ x1 < · · · < xk ≤ X . The interesting result is thatby truncating the sum in (10) at any Sx, a lower or upperbound is obtained, depending on the sign of the last term.Specifically, PU ≤ S1, PU ≥ S1−S2, PU ≤ S1−S2+S3 andso on. Two specific versions of the Bonferroni-type bounds arebriefly described next.

The Kounias lower bound is given by,

PU ≥ maxΓ

⎡⎣∑

x∈Γ

P (Ax) −∑

y,x∈Γ,y<x

P (Ay ∩ Ax)

⎤⎦ , (11)

where Γ is any arbitrary subset of IX = {1, 2, . . . , X}.As pointed out in [18], one can apply an efficient stepwisealgorithm, whose complexity is linear in the number of events,to find the best subset that gives the tightest bound.

For X ≥ 2, the tightest Hunter upper bound is:

PU ≤N∑

x=1

P (Ax) − maxT0∈T∑

(x,y)∈T0

P (Ax ∩ Ay), (12)

where T0 is any tree spanning the indices of the setA1, . . . , AX , (x, y) is an edge in T0. The greedy algorithm[18] can then be applied to construct the optimal spanningtree T0 for (12).

The Bonferroni-type bounds have been applied to ana-lyze the performance of OSTBCs in various scenarios [19],[20], but without antenna selection. In the following, theBonferroni-type bounds are applied to derive the tight upperand lower bounds on the SER and BER of OSTBCs withantenna selection.

A. Bonferroni-Type Bounds on the SERIn general, the SER can be expressed as [19]:

Psymbol(error) =M∑

u=1

P (ε|su)P (su) =1

M

∑u=1

Pu

⎛⎝ ⋃

p �=u

εp,u

⎞⎠ ,

(13)

where P (ε|su) is the conditional error probability giventhat su was sent and εp,u is the event that sp has a largermetric than su [19]. In order to obtain the two Bonferroni-type bounds for the SER, one needs to compute the pairwiseerror probability (PEP), Pu(εp,u), and the two dimensionalPEP, Pu(εp,u ∩ εq,u), for given q, p, and u.

Given Y , the conditional PEP can be computed as follows[19]:

Pu(εp,u|Y ) =1π

∫ π/2

0

exp

(− δ2

p,uY

2 sin2 θ

)dθ (14)

where δp,u =√

Rρ/2Ntdp,u and dp,u = ‖su−sp‖. Averaging(14) over Y , one has:

Pu(εp,u) =1π

∫ π/2

0

φY

(δ2p,u

2 sin2 θ

)dθ (15)

where φY (·) is either φ(r)Y (·) given in (7) for receive antenna

selection, or φ(r,t)Y (·) given in (28) for joint receive and

transmit antenna selection. Clearly, the unconditional PEPPu(εp,u) can be easily calculated using finite-range singleintegrals.

Following the analysis in [18], [19], one has the conditionaltwo-dimensional PEP expressed in terms of two-dimensionaljoint Gaussian function as follows:

Pu(εp,u ∩ εq,u|Y ) = Ψ(ρp,q,u, δpu

√Y , δqu

√Y)

, (16)

where ρp,q,u =< sp − su, sq − su > /dp,udq,u and Ψ(r, a, b)is the two-dimensional joint Gaussian function, defined as:

Ψ(r, a, b) =1

2π√

1 − r2

∫ ∞

a

∫ ∞

b

exp

[x2 − 2rxy + y2

2(1 − r2)

]dxdy.

(17)

Furthermore, since δpu

√Y and δqu

√Y are non-negative, one

can use the single integral representation of (17) to obtain theconditional PEP:

Ψ(ρp,q,u, δpu

√Y , δqu

√Y)

=12π

∫ ϕ(dp,u/dq,u,ρp,q,u)

0

exp

(− δ2

p,uY

2 sin2 θ

)dθ

+12π

∫ ϕ(dq,u/dp,u,ρp,q,u)

0

exp

(− δ2

q,uY

2 sin2 θ

)dθ (18)

where ϕ(x, r) = tan−1(x√

1 − r2/(1 − rx)). Taking the

average of (18) over Y using the MGF of Y , one arrives at:

Pu(εp,u ∩ εq,u) =12π

∫ ϕ(dp,u/dq,u,ρp,q,u)

0

φY

(δ2p,u

2 sin2 θ

)dθ

+12π

∫ ϕ(dq,u/dp,u,ρp,q,u)

0

φY

(δ2q,u

2 sin2 θ

)dθ

(19)

Clearly, the two-dimensional PEP Pu(εp,u ∩ εq,u) can also beeffectively computed by single finite-range integrals. Using(15) and (19), one obtains the lower and upper bounds, basedon the Kounias and Hunter bounds, respectively, for the SER,which shall be referred to simply as the stepwise lower boundand the greedy upper bound. It should be noted that thesebounds are applicable for any arbitrary constellations.

B. Bonferroni-Type Bounds on the BER

The upper and lower bounds on the BER can also beobtained using the MGF of Y . In particular, the BER canbe computed as follows [18], [19]:

Pbit(error) =1M

M∑u=1

Pb(su), (20)

where Pb(su), the bit error probability when su is sent, is

Pb(su) =1b

M∑q=1

H(q, u)

⎡⎣1 − Pu

⎛⎝⋃

p�=q

εp,q

⎞⎠

⎤⎦ . (21)

IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 7, NO. 12, DECEMBER 2008 4821

In (21), H(q, u) is the Hamming distance between the labelof sq and su (which depends on the mapping rule ξ). GivenY , the one-dimensional PEP is computed as [18], [19]:

Pu(εp,q|Y ) = Q(γp,q,u

√Y)

=1π

∫ π/2

0

exp

(−γ2

p,q,uY

2 sin2 θ

)dθ

(22)

where γp,q,u =δ2

p,u−δ2q,u

δp,q. Similar to the previous sections,

averaging (22) over Y and using the MGF of Y , one obtains:

Pu(εp,q) =1π

∫ π/2

0

φY

(γ2

p,q,u

2 sin2 θ

)dθ. (23)

It should be mentioned that this computation is only validwhen γp,q,q ≥ 0. When γp,q,u < 0, one can use therelationship Q(x) = 1−Q(−x) to obtain a similar expression.

On the other hand, given Y , the two-dimensional PEP,Pu(εp,q ∩ εl,q) is given as [18]:

Pu(εp,q ∩ εl,q|Y ) = Ψ(ρp,l,q, γp,q,u

√Y , γl,q,u

√Y)

. (24)

Therefore, the unconditional two-dimensional PEP can beexpressed in the following closed form:

Pu(εp,q ∩ εl,q)

=12π

∫ ϕ(γp,q,u/γl,q,u,ρp,l,q)

0

φY

(γ2

p,q,u

2 sin2 θ

)dθ

+12π

∫ ϕ(γl,q,u/γp,q,u,ρp,l,q)

0

φY

(γ2

l,q,u

2 sin2 θ

)dθ (25)

The expression of Ψ (r, a, b) in (18) is only valid for non-negative a and b. When at least one of a and b is negative,Ψ (r, a, b) can be expressed in the form of other Ψ(·) functionwith non-negative arguments [19]. Hence, a similar expressionof Pu(εp,q ∩ εl,q) can still be obtained

Using (23) and (25), the Kounias and Hunter bounds canbe calculated, which lead to the upper and lower bounds,respectively, for the BER in (20). They are simply referredto as the stepwise upper bound and the greedy lower bound.

IV. ILLUSTRATIVE RESULTS

This section provides the numerical results to confirm ouranalysis. In particular, the cases of receive antenna selectionand joint antenna selection at the transmitter and the receiverare considered. Furthermore, we concentrate on Alamouti

scheme [3] with Nt = N = 2 and G =(

v1 v2

−v�2 v�

1

).

Unless stated otherwise, the number of receive antennas isalways Lr = 6. In the case of receive antenna selection only,one has Lt = Nt = 2. For joint receive and transmit antennaselection, the number of transmit antenna is assumed to beLt = 4. The SNR is defined as ρ (dB).

A. Receive antenna selection

1) SER Performance: Figures 1 and 2 show the simulationresults and the exact analysis of the SER presented in (8)and (9) for the Alamouti scheme employing 8-PSK and 16-QAM constellations, respectively. Three values of Nr, namelyNr = 1, Nr = 2, and Nr = 6, are investigated. For

0 5 10 15 20 25 3010

−5

10−4

10−3

10−2

10−1

100

ρ (dB)

SER

Simulation, 8−PSKExact analysis, 8−PSKGreedy UB, 8−PSKStepwise LB, 8−PSK

Lr=6,N

r=2

Lr=2,N

r=2

Lr=6,N

r=1

Lr=6,N

r=6

Fig. 1. The simulation results, exact analysis, greedy upper bound (UB),and stepwise lower bound (LB) for 2 × 2 Alamouti scheme using 8-PSKconstellation with receive antenna selection for Lt = Nt = 2 and variousvalues of Nr . The performances with Lr = Nr = 2 are provided asreferences.

5 10 15 20 25 30 3510

−5

10−4

10−3

10−2

10−1

100

ρ (dB)

SER

Simulation, 16−QAMExact analysis, 16−QAMGreedy UB, 16−QAMStepwise LB, 16−QAM

Lr=2,N

r=2

Lr=6,N

r=1

Lr=6,N

r=2

Lr=6,N

r=6

Fig. 2. The simulation results, exact analysis, greedy upper bound (UB),and stepwise lower bound (LB) for 2 × 2 Alamouti scheme using 16-QAMconstellation with receive antenna selection for Lt = Nt = 2 and variousvalues of Nr . The performances with Lr = Nr = 2 are provided asreferences.

comparison, the derived greedy upper bound and the stepwiselower bound are provided to show the tightness of the bounds.The performances of the system with Lt = Nt = Lr =Nr = 2 antennas (without antenna selection) are also plottedas references. It can be observed from Figs. 1 and 2 thatthe simulation results, the exact analysis and the boundscannot be distinguished over a wide range of SNR for bothconstellations. This means that the bounds can be used asan effective tool to predict the actual SER of OSTBCs withantenna selection over keyhole fading channels. The resultsagree with [18], [19] that the two Bonferroni-type lower andupper bounds are very accurate in estimating the probabilityof a union.

4822 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 7, NO. 12, DECEMBER 2008

0 5 10 15 20 25 3010

−5

10−4

10−3

10−2

10−1

100

ρ (dB)

SER

Simulation, (1,7)Greedy UB, (1,7)Stepwise LB, (1,7)

Lr=2,N

r=2

Lr=6,N

r=1

Lr=6,N

r=6

Lr=6,N

r=2

Fig. 3. The simulation results, greedy upper bound (UB), and stepwiselower bound (LB) for 2 × 2 Alamouti scheme using (1,7) constellation withreceive antenna selection for Lt = Nt = 2 and various values of Nr . Theperformances for the case of Lr = Nr = 2 are provided as references.

0 5 10 15 20 25 3010

−5

10−4

10−3

10−2

10−1

100

ρ (dB)

BE

R

Simulation, natural mappingGreedy LB, natural mappingStepwise UB, natural mappingSimulation, Gray mappingGreedy LB, Gray mappingStepwise UB, Gray mapping

Lr=6,N

r=2 L

r=6,N

r=1

Lr=2,N

r=2

Fig. 4. The simulation results, greedy lower bound (LB), and stepwise upperbound (UB) on the BER for 2×2 Alamouti scheme using 8-PSK constellationwith receive antenna selection, employing Gray and natural mappings forLt = Nt = 2 and various values of Nr . The performances for the case ofLr = Nr = 2 are provided for comparison.

As mentioned before, the exact computations of the SERin (8) and (9) are only applicable for standard M -PSK andsquare M -QAM. The bounds, on the other hand, do not havethis limitation, since they can be applied for arbitrary constel-lations. As an example, Fig. 3 plots the simulation results, thegreedy UB, and the stepwise LB for 2 × 2 Alamouti schemeusing (1,7) constellation [21] and with Lt = Nt = 2, Lr = 6,and various values of Nr. It can be seen that the bounds andthe simulation results almost coincide, which means that theycan practically provide the exact SER. Similar results werealso observed with different non-regular constellations.

2) BER Performance: Figure 4 plots the simulation results,the greedy lower bound (LB), and the stepwise upper bound

0 5 10 15 20 25 3010

−5

10−4

10−3

10−2

10−1

100

ρ (dB)

SER

Simulation, 8−PSKExact analysis, 8−PSKGreedy UB, 8−PSKStepwise LB, 8−PSK

Lt=4,N

t=2,

Lr=6,N

r=1

Lt=2,N

t=2,

Lr=6,N

r=1

Lt=2,N

t=2,

Lr=6,N

r=2

Lt=4,N

t=2,

Lr=6,N

r=2

Fig. 5. The simulation results, exact analysis, greedy upper bound (UB),and stepwise lower bound (LB) for 2 × 2 Alamouti scheme using 8-PSKconstellation with joint receive and transmit antenna selection for Lt = 4,Nt = 2, and various values of Nr selected from Lr = 6 receive antennas.The performances with receive antenna selection only for Lt = Nt = 2 areprovided as references.

(UB) on the BER for the 2 × 2 Alamouti scheme with 8-PSK constellation that employs Gray and natural mappingsand with Lt = Nt = 2, Lr = 6. Two cases of Nr = 1and Nr = 2 are examined. The bounds for the case ofLt = Nt = Lr = Nr = 2 antennas (i.e., without antennaselection) are also provided for comparison. To the authors’best knowledge, general analytical evaluation of the BER ofOSTBCs with antenna selection under keyhole fading is notavailable in the literature. Similar to the results obtained forthe SER performance, it can be observed that the boundspractically yield the exact BER over a wide range of the SNR,for both Gray and natural mappings. Again, it should be notedthat the bounds are applicable for arbitrary constellations andmappings.

B. Joint receive and transmit antenna selection

1) SER Performance: To further confirm the accuracy ofour analysis in the case of joint receive and transmit antennaselection, Fig. 5 shows the simulation results and the exactanalysis of the SER presented in (8) for the Alamouti schemethat employs 8-PSK constellation. In particular, shown in Fig.5 are the performances for the cases of Lt = 4, Nt = 2,Lr = 6, and various values of Nr. The two derived lower andupper bounds are also plotted for comparison. Furthermore,the performances of the systems performing receive antennaselection only with Lt = Nt = 2, Lr = 6, and differentvalues of Nr presented earlier are provided as references. Asin the case of receive antenna selection, similar observationsregarding the tightness of the greedy upper bound and thestepwise lower bound can be made from Fig. 5. Given thesame number of selected transmit antennas Nt and selectedreceive antennas Nr, the performance advantage of jointtransmit and receive antenna selection over receive antennaselection only can also be clearly observed.

IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 7, NO. 12, DECEMBER 2008 4823

0 5 10 15 20 25 3010

−5

10−4

10−3

10−2

10−1

100

ρ (dB)

BE

R

Simulation, natural mappingGreedy LB, natural mappingStepwise UB, natural mappingSimulation, Gray mappingGreedy LB, Gray mappingStepwise UB, Gray mapping

Lt=2,N

t=2,

Lr=6,N

r=1

Lt=4,N

t=2,

Lr=6,N

r=1

Fig. 6. The simulation results, greedy lower bound (LB), and stepwiseupper bound (UB) on the BER for 2 × 2 Alamouti scheme using 8-PSKconstellation and Gray and natural mappings, with joint receive and transmitantenna selection for Lt = 4, Nt = 2, and Nr = 1 selected from Lr = 6receive antennas. The performances for the case of receive antenna selectiononly with Lt = Nt = 2 are provided for comparison.

2) BER Performance: Finally, Fig. 6 provides the simula-tion results, the greedy lower bound (LB), and the stepwiseupper bound (UB) on the BER for the 2×2 Alamouti schemewith 8-PSK constellation that employs Gray and naturalmappings and for joint receive and transmit antenna selectionwith Lt = 4, Nt = 2, Lr = 6. As before, two differentvalues of Nr, namely Nr = 1 and Nr = 2 are considered.The bounds for the case of receive antenna selection only withLt = Nt = 2 are also provided for comparison. The resultsshown in Fig. 6 again validate our analytical analysis on the biterror probability for the general case of joint antenna selectionat both the transmitter and the receiver.

V. CONCLUSION

This paper investigated the SER and BER of OSTBCs overkeyhole fading channels with antenna selection. For standardM -PSK and square M -QAM constellations, closed-form ex-pressions with finite-range single integrals were provided tocompute exactly the SER. Furthermore, very tight lower andupper bounds on the SER and the BER were developed.These bounds are applicable for arbitrary constellations andmappings and can provide practically the exact SER and BERover a wide range of the SNR. The studies in this papertherefore provide a very effective tool to predict the SER andBER performances of OSTBCs over keyhole fading channelswith antenna selection, without the need of time-consumingsimulations.

APPENDIX ATHE PDF AND MGF OF Y FOR JOINT RECEIVE AND

TRANSMIT ANTENNA SELECTION

Using (6) and based on the expression of the modifiedBessel function of the second kind, namely Kν(x · z) =zν

2

∫ ∞0 exp

[−x2 (v + z2/v)

]v(−ν+1)dv [22], the pdf of Y for

the general case of joint receive and transmit antenna selectionis given as:

1

2(

LrNr

)(LtNt

)pY (y) =1

Γ(Nr)Γ(Nt)y

Nt+Nr2 −1KNr−Nt(2

√y)

+1

Γ(Nr)

Lt−Nt∑k=0

βk

[(Nt + k

Nty

)Nr−12

KNr−1

(2

√y

Nt + k

Nt

)

−Nt−1∑p=0

ηk,pyp+Nr

2 −1Kp−Nr (2√

y)

]

+1

Γ(Nt)

Lr−Nr∑i=0

βi

⎡⎣(

Nr + i

Nry

)Nt−12

KNt−1

(2

√y

Nr + i

Nr

)

−Nr−1∑j=0

ηi,jyj+Nt

2 −1Kj−Nt (2√

y)

]

+

Lr−Nr∑i=0

Lt−Nt∑k=0

βiβkK0

(2

√y

(Nt + k

Nt

)(Nr + i

Nr

))

−(

Lr−Nr∑i=0

Lt−Nt∑k=0

Nt−1∑p=0

βiβkηk,p

(Nr + i

Nry

) p−12

·Kp−1

(2

√y

Nr + i

Nr

))

−(

Lt−Nt∑k=0

Lr−Nr∑i=0

Nr−1∑j=0

βkβiηi,j

(Nt + k

Nty

) j−12

·Kj−1

(2

√y

Nt + k

Nt

))

+

Lr−Nr∑i=0

Nr−1∑j=0

Lt−Nt∑k=0

Nt−1∑p=0

βiηi,jβkηk,pyp+j2 −1Kp−j (2

√y) .

(26)

By using [22, 6.643.3] and [16, 13.1.33], one obtains thefollowing integral regarding the modified Bessel function ofthe second kind:

∫ ∞

0

yμ−1/2exp(−sy)K2ν (2β√

y) dy

=12Γ(μ + ν + 1/2)Γ(μ − ν + 1/2)β2νs−(μ+ν+1/2)

· U(

μ + ν + 1/2, 1 + 2ν,β2

s

), (27)

From (26) and (27) and after some manipulations, the MGFof Y can be computed as follows:

φ(r,t)Y (s)(

Lr

Nr

)(Lt

Nt

) = s−NrU (Nr, Nr − Nt + 1, 1/s)

+Lt−Nt∑

k=0

βk

[(Nt + k

Nt

)Nr−1

s−NrU(

Nr, Nr,Nt + k

Nts

)

−Nt−1∑p=0

ηk,pΓ(p)s−pU (p, 1 + p − Nr, 1/s)

]

4824 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 7, NO. 12, DECEMBER 2008

+Lr−Nr∑

i=0

βi

[(Nr + i

Nr

)Nt−1

s−NtU(

Nt, Nt,Nr + i

Nrs

)

−Nr−1∑j=0

ηi,jΓ(j)s−jU (j, 1 + j − Nt, 1/s)

⎤⎦

+Lr−Nr∑

i=0

Lt−Nt∑k=0

βiβks−1U(

1, 1,(Nr + i)(Nt + k)

NrNts

)

−(

Lr−Nr∑i=0

Lt−Nt∑k=0

Nt−1∑p=0

βiβkηk,pΓ(p)(

Nr + i

Nr

)p−1

s−p

·U(

p, p,(Nr + i)

Nrs

))

−⎛⎝Lt−Nt∑

k=0

Lr−Nr∑i=0

Nr−1∑j=0

βkβiηi,jΓ(j)(

Nt + k

Nt

)j−1

s−j

·U(

j, j,(Nt + k)

Nts

))

+

⎛⎝Lr−Nr∑

i=0

Nr−1∑j=0

Lt−Nt∑k=0

Nt−1∑p=0

βiηi,jβkηk,pΓ(j)Γ(p)s−p

·.U(

p, 1 + p − j,1s

)).

(28)

REFERENCES

[1] V. Tarokh, N. Seshadri, and A. R. Calderbank, “Space-time codes forhigh data rate wireless communication: Performance criterion and codeconstruction,” IEEE Trans. Inform. Theory, vol. 44, pp. 744–765, Mar.1998.

[2] V. Tarokh, H. Jafarkhani, and A. R. Calderbank, “Space-time blockcodes from orthogonal designs,” IEEE Trans. Inform. Theory, vol. 45,pp. 1456–1467, July 1999.

[3] S. M. Alamouti, “A simple transmit diversity technique for wirelesscommunications,” IEEE J. Select. Areas Commun., vol. 16, pp. 1451–1458, Oct. 1998.

[4] S. Sanayei and A. Nosratinia, “Antenna selection in MIMO systems,”IEEE Trans. Commun., vol. 42, pp. 68–73, Mar. 2004.

[5] D. J. Love, “On the prebability of error of antenna-subset selection withspace-time block code,” IEEE Trans. Commun., vol. 53, pp. 1799–1803,Nov. 2005.

[6] S. Kaviani and C. Tellambura, “Closed-form BER analysis for antennaselection using orthogonal space-time block codes,” IEEE Commun.Lett., vol. 10, pp. 704–706, 2006.

[7] D. Gesbert, H. Bolcskei, D. A. Gore, and A. J. Paulraj, “Outdoor MIMOwireless channels: models and performance prediction,” IEEE Trans.Commun., vol. 50, pp. 1926–1934, Dec. 1997.

[8] D. Chizhik, G. J. Foschini, M. J. Gans, and R. A. Valenzuela, “Key-holes, correlations, and capacities of multielement transmit and receiveantennas,” IEEE Trans. Wireless Commun., vol. 1, pp. 361–368, Apr.2002.

[9] S. Sanayei and A. Nosratinia, “Antenna selection in keyhole channels,”IEEE Commun. Mag., vol. 55, pp. 404–408, Mar. 2007.

[10] L. Yang and J. Qin, “Performance of STBCs with antenna selection:spatial correlation and keyhole effects,” IEE Proc. Commun., vol. 153,pp. 15–20, Feb. 2006.

[11] P. Garg, R. K. Mallik, and H. M. Gupta, “Performance analysis of space-time coding with imperfect channel estimation,” IEEE Trans. WirelessCommun., vol. 4, pp. 257–265, Jan. 2005.

[12] P. Garg, R. K. Mallik, and H. M. Gupta, “Exact error performance ofsquare orthogonal space- time block coding with channel estimation,”IEEE Trans. Commun., vol. 54, pp. 430–437, Mar. 2006.

[13] R. K. Mallik and Q. T. Zhang, “A tight upper bound on the PEP ofa space-time coded system,” IEEE Trans. Wireless Commun., vol. 6,pp. 3238–3247, 2007.

[14] S. Sandhu and A. Paulraj, “Space-time block codes: a capacity perspec-tive,” IEEE Commun. Lett., vol. 4, pp. 384–386, Dec. 2000.

[15] A. Muller and J. Speidel, “Capacity of multiple-input multiple-outputkeyhole channels with antenna selection,” in Proc. European WirelessConference, Paris, France, Apr. 2007.

[16] M. Abramowitz and I. A. Stegun, Handbook of Mathematical Functions.New York: Dover Publications, Inc., 1965.

[17] M. K. Simon and M.-S. Alouini, Digital Communications over FadingChannels: A Unified Approach to Performance Analysis. New York:Wiley, 2000.

[18] H. Kuai, F. Alajaji, and G. Takahara, “Tight error bounds for nonuniformsignaling over AWGN channels,” IEEE Trans. Inform. Theory, vol. 46,pp. 2712–2718, Nov. 2000.

[19] F. Behnamfar, F. Alajaji, and T. Linder, “Tight error bounds for space-time orthogonal block codes under slow Rayleigh flat fading,” IEEETrans. Commun., vol. 53, pp. 952–956, 2005.

[20] N. H. Tran, H. H. Nguyen, and T. Le-Ngoc, “Performance bounds of or-thogonal space-time block codes over keyhole Nakagami-m channels,”IEEE Signal Processing Lett., vol. 14, pp. 605–608, Sept. 2007.

[21] N. H. Tran and H. H. Nguyen, “Signal mappings of 8-ary constellationsfor bit interleaved coded modulation with iterative decoding,” IEEETrans. Broadcasting, vol. 52, pp. 92–99, Mar. 2006.

[22] I. S. Gradshteyn and I. M. Ryzhik, Table of Integrals, Series, andProducts. Academic Press, 2000.