post hoc analysis - krigolson teaching · basic logic recall that anova tests the following null...
TRANSCRIPT
BasicLogic
RecallthatANOVAteststhefollowingnullhypothesis:
μ1 =μ2 =μ3
Thus,ifitisrejected,alltheANOVAtellsyouisthatthereisadifference.WeuseCONTRASTSorPOSTHOCcomparisonstodeterminewhichgroups
actuallydiffer.
HowTheyDiffer
- PostHoccomparisonscomparegroups- Contrastspartitionthevarianceinthemodel
- Contrastsareusedtotesthypotheses- PostHoctestsareusedwhenyoudonothaveahypothesis
BasicLogic
Sowhynotjustrunaseriesoft-tests?
μ1 vs μ2,μ2=μ3,μ1 vs μ3
Becausewehavetotalkaboutthe“familywiseerrorrate”becausewearemakingaseriesofcomparisons.
I’veheardthatIamsafewiththreecomparisons,butnotwithfour…
μ1 =μ2 =μ3 weonlyhaveonecasewherewecanmakeaTypeIerror
Thisisalsotruefor
μ1 ≠μ2 =μ3
μ1 =μ2 ≠μ3
μ1 ≠μ2 ≠μ3
BUT…
I’veheardthatIamsafewiththreegroups,butnotfour…
μ1 =μ2 ANDμ3=μ4
HerewehaveTWOchancestomakeaTypeIerror,thusourfamilywiseerrorrateisactually0.1,not0.05
Thus,weshouldadjustfortheINFLATIONofTypeIerror.
Fisher’sLeastSignificantDifferenceTest
€
t =X1 − X 2
MSerror (1n1
+1n2)
Alsoknownasaprotectedt-test(wehavereplacedtheestimateofpooledvariancewithMSerror)
But,thisisinrealitystilljustat-testsoitisveryliberal(i.e.,notyourbestchoice).
OtherCommonPosthoc Tests
Bonferroni:Essentiallyadjustsalphatocorrectformultiplecomparisonsandthent-testsareused.
alphanew =alpha/numberoftests
OtherCommonPosthoc Tests
Tukey’s:Reflectsanadjustmenttotheactualformulaeusedtogeneratethetstatistic,andcorrectsformultiplecomparisonsthisway.
Foralargenumberofcomparisons,Tukey’s TestismorepowerfulthanBonferonni.Forasmallnumberofcomparisons,viceversa.
OtherCommonPosthoc Tests
Hochberg’sGT2andGabriel’sPairwiseComparisonsTestsaresimilartoTukey’s Test.
Hochberg’sTestsisgenerallymoreliberal.Gabriel’sPairwiseTestsaregoodwhencellsizesareunequal.
OtherCommonPosthoc Tests
Dunnett’s PairwiseTestscompareallofthegroupmeansagainstacontrolmean(thefirstofthelast).Mostlyusedwhencomparingclinicalpopulationstoacontrolgroup.
OtherCommonPosthoc Tests
Tamhane’s T2,Dunnett’s T3,Games-Howell,andDunnett’s Careallteststobeusedwhenthevariancesareunequal.
OtherCommonPosthoc Tests
Waller-DuncanTest:ABayesian(probability)approach.
Scheffe Test:Generallythemostconservative,examinesalllinearcombinationsofvariables.
SimpleEffectsAnalysis
SimpleEffectsAnalysisdenotesonlyusingt-tests,butusingalogicalcomparison.IE,lookattheplotanddonotcomparethingsthatdonotmakesense.IfGroup2>Group1,andGroup3>Group2,whycompareGroup1andGroup3?
SowhatdoIdo…
1.Decideifyouaregoingtouseacontrastapproachoraposthocapproach.
2.Ifyouuseacontrastapproach,thekeyconcepttogetisthatyourcontrastsMUSTbelogicallydefined.
3.Ifyouuseaposthocapproach,thekeyconcepttogetisthatifyouaremakingmanycomparisons(morethan3)youneedtodosomething,anythingreally,toadjustforthis.
Contrasts
Essentiallycontrastsbreakdownthevarianceaccountedforbythemodeltoseewherethe
differenceslie.
RulesWhenUsingContrasts
1. Choosesensiblecomparisons,youonlywanttocomparetwo“chunks”ofvariancewitheachcomparison.Further,ifyousingleoutasinglegroupinacomparison,onlyuseitinonecomparisonandnosubsequentcomparisons.
RulesWhenUsingContrasts
2.Weightsareusedtosetupcontrasts.Ifasked,youmustdefinetheweightstosumtozero.
ContrastGroupA: +1ContrastGroupB: - 1
RulesWhenUsingContrasts
3.Groupsnotinacontrastshouldbegivenaweightof0.Why?0xGroupVariance=0soitisnegatedfromthecontrast.
TypesofContrastsDeviation:
Deviationfromoverallmeanwith1groupomitted.(generallynotused)
Simple:Level2vs Level1,Level3vs Level1,etc…(orlast)
Difference:Level2vs Level1,Level3vs Previous,etc…
Helmert:Level1vs Later,Level2vs Later,etc…
Repeated:Contrasts1to2,2to3,3to4,etc…
Polynomial:Looksforalinear,quadratic,orcubictrend.