PARETO DEPTH SAMPLING DISTRIBUTIONS FOR GENE RANKING

G. Fleury

Ecole Superieure d’ElectriciteService des Mesures,

91192 Gif-sur-Yvette, France

A. Hero†, S. Zareparsi∗ and A. Swaroop∗

University of Michigan†Dept. of EECS and ∗Dept. of Ophthalmology and Visual Sciences

Ann Arbor MI, USA

ABSTRACT

In this paper we propose a method for gene rankingfrom microarray experiments using multiple discriminants.The novelty of our approach is that a gene’s relative rank isdetermined according to the ordinal theory of multiple ob-jective optimization. Furthermore, the distribution of eachgene’s rank, called Pareto depth, is determined by resam-pling over the microarray replicates. This distribution iscalled the Pareto depth sampling distribution (PDSD) andit is used to assess the stability of each ranking. Graphicalrepresentation of the PDSD as an image communicates in-formation about the stability of each gene’s rank. We illus-trate on data from a mouse retina microarray experiment0.

1. INTRODUCTION

Multicriteria gene filtering seeks to find genes whose ex-pression profiles strike an optimal compromise between max-imizing (or minimizing) several criteria. It is often easier fora molecular biologist to specify several criteria than a singlecriterion. For example the biologist might be interested inaging genes, which he might define as those genes havingexpression profiles that are increasing over time, have lowcurvature over time, and whose total increase from initialtime to final time is large. Or one may have to deal with twobiologists who each have different criteria for what featuresconstitute an interesting aging gene.

In this paper we present a general method for rank or-dering of genes based on a statistical version of the Paretofront partial order in multicriteria optimization. As a linearordering of multiple criteria does not generally exist, an ab-solute ranking of the selected genes is generally impossible.However a partial ordering is often possible when formu-lated as a multicriterion optimization problem. This ideawas used in our previous work [1, 2, 3] to obtain relativerankings of gene expression levels based on microarray ex-periments. We called our multiobjective approach to gene

0This research was partially supported by National Institutes of Healthgrant NIH-EY11115 (including microarray supplements), Macula VisionResearch Foundation, and Elmer and Sylvia Sramek Foundation.

ranking Pareto front analysis (PFA). As pointed out in [3]the PFA approach is related to the notion of data depths andcontours of depth in a multivariate sample [4]. In an analo-gous manner we will refer to the Pareto depth of a gene asthe Pareto front on which the gene lies. Here we introducethe Pareto depth sampling distribution (PDSD) as a tool toboth select high ranked genes and to visualize the stabilityof the gene rankings as an image. Gene microarray datafrom two experiments will be used to illustrate our analysis.Mouse Retinal Aging Study: The experiment consists ofhybridizing 24 retinal tissue samples taken from each of24 age-sorted mice at 6 ages (time points) with 4 repli-cates per time point. These 6 time points consisted of 2early development (Pn2, Pn10) and 4 late development (M2,M6, M16, M21) samples. RNA from each sample of reti-nal tissue was reverse transcribed to cDNA, amplified andhybridized to the 12,422 probes on one of 24 AffymetrixU74Av2 Mouse GeneChip microarrays. The data arraysfrom the GeneChips were processed by Affymetrix MAS5software to yield log2 probe response data.Human Retinal Aging Study: The experiment consists ofhybridizing 16 retinal tissue samples taken from 8 younghuman donors and 8 old human donors. The ages of theyoung donors ranged from 16 to 21 years and the ages ofthe old donors ranged from 70 to 85 years old. The 16 tis-sue samples were hybridized to 16 Affymetrix U95A Hu-man GeneChip microarrays each containing N = 12, 642probes.

2. GENE SCREENING AND RANKING

We assume that there are T populations (time samples) eachconsisting of Mt replicates, t = 1, . . . , T . For each of thesamples we assume an independent microarray hybridiza-tion experiment is performed yielding N gene probe re-sponses extracted from the microarray. Define the measuredresponse of the n-th probe on the m-th microarray acquiredat time t

ytm(n), n = 1, . . . , N, m = 1, . . . , M, t = 1, . . . , T.

650-7803-8388-5/04/$20.00 ©2004 IEEEAuthorized licensed use limited to: University of Michigan Library. Downloaded on May 6, 2009 at 09:48 from IEEE Xplore. Restrictions apply.

Consider the common problem of finding a set of geneswhose mean expression levels are significantly different be-tween a pair of time points (T = 2) [5]. The measured proberesponses from such genes should exhibit small variabilityover population (intra-class dispersion) and high variabil-ity over time (inter-class dispersion). Two natural measuresof intra-class dispersion ξ1 and inter-class dispersion ξ2,respectively, are the (scaled) absolute difference betweensample means:

ξ2(n) =1√

1M1

+ 1M2

|y1.(n) − y2.(n)| , (1)

where yt.(n) = M−1t

∑Mt

m=1 ytm(n), and the pooled sam-ple standard deviation:

ξ1(n) =

√(M1 − 1)σ2

1(n) + (M2 − 1)σ22(n)

(M1 − 1) + (M2 − 1)(2)

where σ2t (n) = (Mt − 1)−1

∑Mt

m=1 (ytm(n) − yt.(n))2 .The simple paired t-test can be used to separate the popu-lations by thresholding the ratio Tpt(n) = ξ2(n)/ξ1(n) ofthe two dispersion measures and this could be used to rankthe genes in decreasing order of Tpt, or, equivalently, in in-creasing p-value.

ξ1

ξ2Optimum

ξ1

ξ2

Fig. 1. Left: a linear ordering exists and a single gene (optimum)dominates the others.Right: No non-trivial partial ordering exists.

Multiple objective optimization captures the intrinsic com-promises among possibly conflicting objectives. To illus-trate, in the present context we consider the pair of crite-ria ξ2(n) (1) and ξ1(n) (2). A gene that maximizes ξ2 andminimizes ξ1 over all genes would be a very attractive geneindeed (Fig. 1.a). Unfortunately, such an extreme of op-timality is seldom attained with multiple criteria. In rarecases there exists no non-trivial partial ordering and no sen-sible ranking is possible (see Fig. 1.b). However, in mostcases, illustrated in Fig. 2.a, a partial ordering is possible.In the left panel of Fig. 2 gene A dominates gene C becauseboth criteria are higher for A than for C. D, A and B are saidto be non-dominated because improvement of one criterion

in going from D to A to B corresponds to degradation ofthe other criterion. All the genes which are non-dominatedconstitute a curve which is called the Pareto front. A sec-ond Pareto front is obtained by stripping off points on thefirst front and computing the Pareto front of the remainingpoints (see Fig. 2). This process can be repeated to definea third front and so on. A gene that lies on the k-th Paretofront will be said to be at ”Pareto depth” k.

A

B

C

D ξ

ξ2

1 ξ

ξ2

1

Fig. 2. Left: A, B, D are non-dominated genes and form the Paretofront in the dual criteria plane where ξ1 is to be minimized and ξ2

is to be maximized. Right: successive Pareto fronts in dual criteriaplane (o : first Pareto front, * : second Pareto front, + : thirdPareto front).

In practical cases there are multiple Pareto fronts each con-sisting of many genes. We illustrate in Figs. 3 and Fig. 4where we show the scatterplots, called sample mean multi-criteria scattergrams, of the empirical criteria {(ξ1(n), ξ2(n))}N

n=1

defined in (1) and (2) for all gene probe responses extractedfrom microarrays in the mouse retina aging experiment andthe human retina aging experiment, respectively.

100

10−3

10−2

10−1

100

ξ1 (intra class : MIN)

ξ 2 (in

ter

clas

s : M

AX

)

Fig. 3. The sample mean multicriterion scattergram for themouse retina aging experiment when comparing the populationsat two time points M21 and M2. The first three Pareto fronts areindicated by circles, squares, and asterisks, respectively.

66Authorized licensed use limited to: University of Michigan Library. Downloaded on May 6, 2009 at 09:48 from IEEE Xplore. Restrictions apply.

100

101

102

103

104

10−2

10−1

100

101

102

103

104

ξ1 (intra class : MIN)

ξ 2 (in

ter

clas

s : M

AX

)

Fig. 4. The sample mean multicriterion scattergram for thehuman retina aging experiment (analog to Fig. 3) whencomparing young to old populations.

3. PARETO DEPTH SAMPLING DISTRIBUTION

To account for sample variation we applied a simple leave-one-out cross-validation procedure to evaluate the sensitiv-ity of the Pareto fronts to resampling the available samples.For each time point a sample is omitted leaving 2M sets of(M − 1)2 pairs to be tested (here we set Mt = M , corre-sponding to the two data sets presented above). For eachof these resampled set of genes the Pareto fronts are com-puted. The most resistant genes are those which remain onthe top Pareto fronts throughout the resampling process. Toquantify the movement of a given gene across the Paretofronts as we resample, we introduce the Pareto depth sam-pling distribution (PDSD). For each gene this distributioncorresponds to the empirical distribution of the Pareto frontindexes visited during the resampling process:

Pdsdn(k) = M−1resamp

Mresamp∑j=1

1n(j, k), k = 1, . . . , N

where Mresamp = 2M is the number of resampling trials,and 1n(j, k) is an indicator function of the event: ”j-th re-sampling of n-th gene is on k-th Pareto front.” If K is the to-tal number of Pareto fronts in the scattergram (ξ1(n), ξ2(n)}N

n=1

then, by convention, we define Pdsdn(k) = 0 for k > K.As the PDSD is a probability distribution Pdsdn(k) ≥ 0and

∑k Pdsdn(k) = 1.

Figure 5 corresponds to the (un-normalized) PDSDs overthe first 40 Pareto depths for four different genes taken fromthe human data set under the dual criteria (ξ1, ξ2) of (1) and(2). The left and right panels of Fig. 6 show the PDSDsof the top 50 genes for the human retina data and mousedata, respectively. In each of these figures the top 50 genes

1 2 3 4 50

20

40

60

PDSD

1 2 3 4 50

20

40

60

1 3 5 7 9 110

20

40

60

Pareto front number

PDSD

1 10 20 30 400

20

40

60

Pareto front number

Fig. 5. Unormalized PDSDs for four different genes takenfrom human retina experiment. These PDSDs are indexedby the Pareto depth, which is equivalent to Pareto front num-ber.

were ranked in order of increasing second PDSD moment∑Kk=1 k2Pdsdn(k). The PDSD images provide graphic in-

dication of the Pareto variability of the human and mousedata sets. We note that even though the human data set hashigher variance than the mouse data set, the top 50 humangenes have lower Pareto variability since the human Paretofronts are broader and contain more genes (compare Figs. 3and 4).

Pareto front number

(m2 +

σ2 ) de

pth

sort

ed g

enes

2 4 6 8 10 12 14 16 18 20

5

10

15

20

25

30

35

40

45

50

Pareto front number

(m2 +

σ2 ) de

pth

sort

ed g

enes

2 4 6 8 10 12 14 16 18 20

5

10

15

20

25

30

35

40

45

50

Fig. 6. Left: An image of the PDSDs of the 50 top Pareto rankedhuman genes. Right: An image of the PDSDs of the 50 top Paretoranked mouse genes. The magnitude of the PDSD is encoded inthe false color range of black (PDSD=1) to white (PDSD = 0).

4. RANKING RATE COMPARISONS

We investigated the ranking performance of the second mo-ment PDSD gene ranking procedure to the ranking perfor-mance of the paired t-test. Three hundred (N = 300) dif-ferent probe responses were simulated. Eight (M = 8)replicates of the n-th gene probe response were generated

67Authorized licensed use limited to: University of Michigan Library. Downloaded on May 6, 2009 at 09:48 from IEEE Xplore. Restrictions apply.

10−5

10−4

10−3

10−2

10−1

100

100

101

102

103

104

105

ξ1 (intra class : MIN)

ξ 2 (in

ter

clas

s : M

AX

)

10−4

10−3

10−2

10−1

100

100

101

102

103

104

105

Estimated ξ1 (intra class : MIN)

Est

imat

edξ 2 (

inte

r cl

ass

: MA

X)

Fig. 7. Left: ensemble mean scattergram (ground truth) for simu-lation study. Right: sample mean scattergram formed from a ran-dom realization.

according to an i.i.d. Gaussian distribution with means andvariances given by (m1(n), σ2

1(n)) and (m2(n), σ22(n)) for

populations 1 and 2, respectively. The variances were madeequal σ2

1(n) = σ22(n) = σ2(n) over both populations. The

means and variances were set by the following formula:

σ(n) = ξ2(n), m1(n) = 0, m2(n) = ξ1(n)ξ2(n)/2.

The values of ξ1(n), ξ2(n) are illustrated in the ground truthscattergram in the left panel of Fig. 7. We designate the 90genes on the first 3 fronts of this figure (depth increasingalong −45o diagonal) as ground-truth-optimal genes.

The right panel of Fig. 7 shows a realization of the em-pirical scattergram obtained from sample mean and varianceestimates derived from the replicates. Figure 8 shows thethree first Pareto fronts and the boundaries of two accep-tance regions for the paired t-test applied to the empiricalscattergram of Fig. 7. The first three Pareto fronts do notcapture all of the ground-truth-optimal genes but they havea very low (0%) false discovery rate (proportion of genesfound which are not ground-truth-optimal). The solid lineboundary of the paired t-test discovers the 90 genes withlowest p-value. Use of this acceptance region would resultin discovery of more ground-truth-optimal genes than dis-covered by the first three Pareto fronts, but with a false dis-covery rate of approximately 15%. The dashed line bound-ary corresponds to a paired t-test threshold which wouldlead to discovery of all of the 90 ground-truth-optimal genes,however, the false discovery rate of this acceptance regionis quite high (> 40%).

In Fig.9 we plot the correct discovery rate and the false dis-covery rate, respectively, for the paired t-test ranking andthe second moment PDSD ranking procedures. The latterPareto depth test performed significantly better (higher cor-rect discovery rate and lower false discovery rate) than thepaired t-test for all M investigated.

5. CONCLUSION

This paper has presented a new method of Pareto analysisthat can identify and rank genes that have both stable and

10−6

10−5

10−4

10−3

10−2

10−1

100

101

10−2

10−1

100

101

102

103

104

105

ξ1 (MIN)

ξ 2 (M

AX

)

Fig. 8. Three first Pareto fronts (circle, square and asterisk)and boundaries of paired t-test acceptance ragions.

1.5 2 2.5 3 3.5 4 4.5 5 5.586

88

90

92

94

96

98

100

log2(number of samples)

Cor

rect

Dis

cove

ry R

ate

(%)

1.5 2 2.5 3 3.5 4 4.5 5 5.50

5

10

15

20

25

30

35

40

45

log2(number of samples)

Fals

e D

isco

very

Rat

e

Fig. 9. Correct discovery rate (left) and false discovery rate(right) as a function of the number of replicates for paired t-test(solid) versus Pareto depth test (dashed).

low Pareto depths relative to the remaining genes. Addi-tional genes discovered using this algorithm are now beingvalidated by RT-PCR methods. The developed method hasbeen implemented in Matlab and C and is sufficiently fastto be part of an interactive tool for gene screening, ranking,and clustering.

6. REFERENCES

[1] G. Fleury, A. O. Hero, S. Yosida, T. Carter, C. Barlow, andA. Swaroop, “Clustering gene expression signals from retinalmicroarray data,” in Proc. IEEE Int. Conf. Acoust., Speech,and Sig. Proc., Orlando, FL, 2002, vol. IV, pp. 4024–4027.

[2] G. Fleury, A. O. Hero, S. Yosida, T. Carter, C. Barlow, andA. Swaroop, “Pareto analysis for gene filtering in microar-ray experiments,” in European Sig. Proc. Conf. (EUSIPCO),Toulouse, FRANCE, 2002.

[3] A. Hero and G. Fleury, “Pareto-optimalmethods for gene analysis,” Journ. of VLSISignal Processing, vol. to appear, 2003,www.eecs.umich.edu/˜hero/bioinfo.html.

[4] J. Tukey, Exploratory Data Analysis, Wiley, NY NY, 1977.

[5] Terry P. Speed, Statistical analysis of gene expression mi-croarray data, CRC Press, 2003.

68Authorized licensed use limited to: University of Michigan Library. Downloaded on May 6, 2009 at 09:48 from IEEE Xplore. Restrictions apply.