IMST 2008 / FIM XVI

Decision theoretic Bayesian hypothesis testing with focus on skewed alternatives

By

Naveen K. BansalRu Sheng

Marquette UniversityMilwaukee, WI 53051

Email: naveen.bansal@mu.edu

http://www.mscs.mu.edu/~naveen/

0:1,0:1)0,(0:0

),P( :Modely Probabilit

HHvssayH

IMST 2008 / FIM XVI

Directional Error (Type III error):

Sarkar and Zhou (2008, JSPI)Shaffer (2002, Psychological Methods)Finner ( 1999, AS)Lehmann (1952, AMS; 1957, AMS)

Main points of these work is that if the objective is to find the true Direction of the alternative after rejecting the null, then a Type III error must be also controlled.

Type III error is defined as P( false directional error if the null is rejected). The traditional method does not control the directional error. For example,

0. if occurserror an , and ,|| 2/2/ tttt

IMST 2008 / FIM XVI

.action gfor takin loss),(},1,0,1{

)(0,in containedsupport ith density w)(1

,0)(-in containedsupport ith density w)(1

)0()(1),0()(0),0()(1)(1-

where

)(11)(00)(11)(

0:1,0:1,)0,(0:0

),P( :Modely Probabilit

AaaLA

g

g

IgIIg

ppp

HHsayH

IMST 2008 / FIM XVI

same. theare error) I (Type ),0( for which rules all of class theconsider

}:))(),0(),({()S(

space thecomparingby compared becan rulesdecision ,prior fixed aFor

),0()0(

)(),()(

)(),()( where

)()0()()(

bygiven is ruledecision afor risk Bayes The

*11011

0

0 111

0 111

11100111

R

Drrr

Rr

dRr

dRr

rprprpr

BayesRule

)(1 r

)(1 r11 and on

depends slopepp

11 pp

Bayes Rule

11 pp

IMST 2008 / FIM XVI

direction. positive in therisk averagehan smaller t be willdirection negative in the

rule Bayes theofrisk average e then th),11( 1an likely th more is 1t known tha isit apriori if that implies theoremThis:

)'('

1)(1

and

)'('

1)(1

thenly,respective ' and under rules Bayes theare 'B and B if and , '

11 If :1

)' tocomparisonin under likelely more is 1 (i.e. '11

)(2'1)(00)(1

'1-)('

),(11)(00)(11-)(

priors twofollowing heConsider t

ppHH

BrBr

BrBr

pp

Hpp

ppp

ppp

Remark

Theorem

IMST 2008 / FIM XVI

Relationship with the False Discovery Rates

s.discoverie false theseof average weighted theis )( rate.dicovery -non false expected as )(

and rate,discovery positive-non false expected as dinterprete becan )(Similarly framwork.Bayesian in the ratediscovery negative-non false expected iswhich

)]1|0([1

)]0|0([)()0()(

0)0(0)0(0)0(

),(

0100

)1,(0100

)0,(01 0 0

,-1)(

loss 1"-"0 heConsider t

0

1

0 11

BB

B

B

rrr

PE

PEdPr

PPP

R

LLL

IMST 2008 / FIM XVI

rule Bayes randomized-non}1,0,1{

tindependen ,...,2,1),(11)(00)(11~

0:1,0:1,0:0

,...,2,1),,;(~i

withdatat independen },...,2,1,{

iB

mipppi

iiHi

iHiiH

miiif

mIi

xX

X

ProblemHupotheses Multiple

An Application:

An experiment was conducted to see the effect of a sequence knockdown from non-coding genes. Hypothesis was that this knockdown will cause the overexpression of the mRNAs of the coding genes. The implication of this is that this will shows how the non-coding genes interact with coding genes. In other words, non-coding genes also play a part in protein synthesis.

Here it can be assumed that that the effect of knockdown on the coding genes (if there is any) would be mostly overexpression of mRNAs than underexpression of mRNAs. The objective would be to detect as many of overexpressed genes as possible.

IMST 2008 / FIM XVI

IMST 2008 / FIM XVI

Accept Accept Accept Total

is true

is true

is true

Total

U 1V 2V 0m

1H1T 1S 2W 1m

1H2T 1W 2S 2m

0R 1R 2R

0H 1H 1H

0H

Table 1Possible outcomes from hypothesis tests

m

m

)2003,(]0|[]0|[

][][

22

2211

1

111

2

221

1

111

ASStoreyRRWVEpFDRR

RWVEpFDR

RWVEFDR

RWVEFDR

IMST 2008 / FIM XVI

)1|1()1|0()(1

)1|1()1|0()(1

then trials,l trinomiaidenticalandt independen are ,...,2,1),1,0,1( If ly.respective 1

and 0,1 wrt of onsdistributi marginal theare 1,0,1

where,1)1(0)0(1)1(~| that Assume

.,...,2,1each for ly,respective ,1 and 1 accepting of

regions thebe 1 and 1Let .,...,2,1 statisticson test based are testshypothesis that Suppose :

THPTHPpFDR

and

THPTHPpFDR

miiHiHiHiTFFF

FiHIFiHIFiHIiidiHiT

miiHiHmTTTm

2 Theorem

IMST 2008 / FIM XVI

This theorem and our previous discussion on the Bayes decision rule implies that under the independent hypotheses testing setting, the Bayes rule obtained from a single test problem

which minimizes the average expected false discoveries, can be used under multiple hypotheses problem. The false discovery rates of this procedure can be obtained by computing the posterior probabilities.

,0:1,0:1 vs.0:0 HHH

2 and 1 pFDRpFDR

)1|1()1|0()(1

)1|1()1|0()(1

BHPBHPBpFDR

BHPBHPBpFDR

The prior probabilities will determine whether the positive hypothesis or the negative hypothesiswill be more frequently correctly detected.

1 and 1 pp1H 1H

IMST 2008 / FIM XVI

Computation of Bayes Rule:

].,1|)1,([ and ],1|)1,([ computing and)|1( and ),|0(),|1( computing invole partsn Computatio

ly.respective )(1 and )(1under of densities posterior theare ),1|( and ),1|( and ly;respective 1 and ,0,1 ofiesprobabilitposterior theare )|1( and ),|0(),|1( Here

),1|()|1()0()|0(),1|()|1()|(

]}|),([1,0,1

min]|),([:)1,0,1({)(

XXXXX

XXXXX

XXXXXX

XX

HLEHLEHHH

HHHHHHHH

HHIHHH

jLEj

iLEiB

IMST 2008 / FIM XVI

))],(/)0,([log()(

is )( of choice good One null. thefrom departure of magnitude hereflect t )( Here,

.constants negative-non some are 1 and 1,0 where

))(1(1)(1)),(1(1)(1 )),((10)(0

be wouldchoice goodA

.1)(1)(1)(0 loss, 1"-"0 For the

0)(1

00)1,(,0)(1

00)1,(,0)(0

00)0,(

XfXfEq

qq

lll

qlqqlqqlq

qqq

qLqLqL

Choice of the loss function:

IMST 2008 / FIM XVI

)1|(1)()1|(1

if )1(1select Otherwise,

.0

1)1|(

)0|( and

0

1)1|(

)0|(

if 0reject ds,other worIn

.under density marginal theis )|( Here

)}1,0,1),|(max()|(:{

bygiven is rule Bayes theloss, 1"-"0 he Under t:

loss. 1"-"0 a have then we,1)(1)(1)( If

HfpHfp

HH

p

p

Hf

Hf

p

p

Hf

Hf

H

jHjHf

jjHfjpiHfipiB

qqq

XX

X

X

X

X

XX

XX

3 Theorem

IMST 2008 / FIM XVI

ondistributi )1,0( half-right:1

ondistributi )1,0( half-left:1

)(11)0(0)(11)(:Prior

0:1,0:1.0:0

)1,( :Modely Probabilit

:Example

N

N

pIpp

HHvsH

N

IMST 2008 / FIM XVI

)1/0(11 and )1/0(1

1

toequivalent isregion rejection theThus ly.respective functions,increasing and decreasinglly monotonica are 1 and 1 that Note

)1

())1(2

22exp(

12)(1

)1

())1(2

22exp(

12)(1

where),(11)()(11 if )1(1select otherwise

,1/0)(1 and 1/0)(1

if 0reject rule Bayes theloss, 1"-"0 he Under t:

ppTxppTx

TT

nnx

nxn

nxT

andnnx

nxn

nxT

xTpxTpHH

ppxTppxT

H

4 Theorem

IMST 2008 / FIM XVI

Power Comparison:

line dotted red : testunbiased UMPline) blue :22.01,39.00,39.01 test withBayes

unbiased. UMPisregion rejection the,11When

ppp

pp

-0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.50

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

mean value

prob

of r

ejec

tion

IMST 2008 / FIM XVI

-0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.50

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

mean value

prob

of r

ejec

tion

line dotted red : testunbiased UMPline) blue :39.01,39.00,22.01 test withBayes ppp

IMST 2008 / FIM XVI

where),(1)|1()()(1)|1E( if )1(1select otherwise

,)|1(

)|0()(1 and

)|1(

)|0()(1

if 0reject rule Bayes theloss, 1"-"0 he Under t:5

result. following get the weproblem, normal For the

ly.respective )|1( and ),|0(),|1(by 1,0,1 replace tobe woulddifference

only e that thsee easy to isIt prior.Dirichlet aconsider can One

xTpExTpHH

pE

pExT

pE

pExT

H

XpEXpEXpEpandpp

XX

X

X

X

X

Theorem

unknown? are 1pand,0p,1p ifWhat

IMST 2008 / FIM XVI

Another Approach:

values.in dispersion the and values,in skewness thereflects Here

).0()()(2)('

normal-skew example,for prior; skewed a is )(' where

),(')01()0(0p)(

prior heConsider t

I

pI

IMST 2008 / FIM XVI

-0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.50

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

mean value

prob

of r

ejec

tion

Power Comparison of a skew-normal Bayes with the UMP test

IMST 2008 / FIM XVI