robust statistics, revisited - people | mit csailpeople.csail.mit.edu/moitra/docs/robust2.pdf ·...
TRANSCRIPT
![Page 1: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/1.jpg)
RobustStatistics,Revisited
AnkurMoitra(MIT)
jointworkwithIlias Diakonikolas,JerryLi,Gautam Kamath,DanielKaneandAlistairStewart
![Page 2: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/2.jpg)
CLASSICPARAMETERESTIMATIONGivensamplesfromanunknowndistributioninsomeclass
e.g.a1-DGaussian
canweaccuratelyestimateitsparameters?
![Page 3: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/3.jpg)
CLASSICPARAMETERESTIMATIONGivensamplesfromanunknowndistributioninsomeclass
e.g.a1-DGaussian
canweaccuratelyestimateitsparameters? Yes!
![Page 4: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/4.jpg)
CLASSICPARAMETERESTIMATIONGivensamplesfromanunknowndistributioninsomeclass
e.g.a1-DGaussian
canweaccuratelyestimateitsparameters?
empiricalmean: empiricalvariance:
Yes!
![Page 5: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/5.jpg)
Themaximumlikelihoodestimatorisasymptoticallyefficient(1910-1920)
R.A.Fisher
![Page 6: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/6.jpg)
Themaximumlikelihoodestimatorisasymptoticallyefficient(1910-1920)
R.A.Fisher J.W.Tukey
Whatabouterrors inthemodelitself?(1960)
![Page 7: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/7.jpg)
ROBUSTSTATISTICS
Whatestimatorsbehavewellinaneighborhood aroundthe model?
![Page 8: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/8.jpg)
ROBUSTSTATISTICS
Whatestimatorsbehavewellinaneighborhood aroundthe model?
Let’sstudyasimpleone-dimensionalexample….
![Page 9: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/9.jpg)
ROBUSTPARAMETERESTIMATIONGivencorrupted samplesfroma1-DGaussian:
canweaccuratelyestimateitsparameters?
=+idealmodel noise observedmodel
![Page 10: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/10.jpg)
Howdoweconstrainthenoise?
![Page 11: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/11.jpg)
Howdoweconstrainthenoise?
Equivalently:
L1-normofnoiseatmostO(ε)
![Page 12: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/12.jpg)
Howdoweconstrainthenoise?
Equivalently:
L1-normofnoiseatmostO(ε) ArbitrarilycorruptO(ε)-fractionofsamples(inexpectation)
![Page 13: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/13.jpg)
Howdoweconstrainthenoise?
Equivalently:
ThisgeneralizesHuber’sContaminationModel:Anadversarycanadd anε-fractionofsamples
L1-normofnoiseatmostO(ε) ArbitrarilycorruptO(ε)-fractionofsamples(inexpectation)
![Page 14: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/14.jpg)
Howdoweconstrainthenoise?
Equivalently:
ThisgeneralizesHuber’sContaminationModel:Anadversarycanadd anε-fractionofsamples
L1-normofnoiseatmostO(ε) ArbitrarilycorruptO(ε)-fractionofsamples(inexpectation)
Outliers:Pointsadversaryhascorrupted,Inliers:Pointshehasn’t
![Page 15: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/15.jpg)
Inwhatnormdowewanttheparameterstobeclose?
![Page 16: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/16.jpg)
Inwhatnormdowewanttheparameterstobeclose?
Definition:Thetotalvariationdistancebetweentwodistributionswithpdfs f(x)andg(x)is
![Page 17: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/17.jpg)
Inwhatnormdowewanttheparameterstobeclose?
FromtheboundontheL1-normofthenoise,wehave:
observedideal
Definition:Thetotalvariationdistancebetweentwodistributionswithpdfs f(x)andg(x)is
![Page 18: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/18.jpg)
Inwhatnormdowewanttheparameterstobeclose?
Definition:Thetotalvariationdistancebetweentwodistributionswithpdfs f(x)andg(x)is
estimate ideal
Goal:Finda1-DGaussianthatsatisfies
![Page 19: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/19.jpg)
Inwhatnormdowewanttheparameterstobeclose?
estimate observed
Definition:Thetotalvariationdistancebetweentwodistributionswithpdfs f(x)andg(x)is
Equivalently,finda1-DGaussianthatsatisfies
![Page 20: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/20.jpg)
Dotheempiricalmeanandempiricalvariancework?
![Page 21: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/21.jpg)
Dotheempiricalmeanandempiricalvariancework?
No!
![Page 22: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/22.jpg)
Dotheempiricalmeanandempiricalvariancework?
No!
=+idealmodel noise observedmodel
![Page 23: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/23.jpg)
Dotheempiricalmeanandempiricalvariancework?
No!
=+idealmodel noise observedmodel
Asinglecorruptedsamplecanarbitrarilycorrupttheestimates
![Page 24: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/24.jpg)
Dotheempiricalmeanandempiricalvariancework?
No!
=+idealmodel noise observedmodel
Asinglecorruptedsamplecanarbitrarilycorrupttheestimates
Butthemedian andmedianabsolutedeviationdowork
![Page 25: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/25.jpg)
Dotheempiricalmeanandempiricalvariancework?
No!
=+idealmodel noise observedmodel
Asinglecorruptedsamplecanarbitrarilycorrupttheestimates
Butthemedian andmedianabsolutedeviationdowork
![Page 26: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/26.jpg)
Fact[Folklore]:Givensamplesfromadistributionthatareε-closeintotalvariationdistancetoa1-DGaussian
themedianandMADrecoverestimatesthatsatisfy
where
![Page 27: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/27.jpg)
Fact[Folklore]:Givensamplesfromadistributionthatareε-closeintotalvariationdistancetoa1-DGaussian
themedianandMADrecoverestimatesthatsatisfy
where
Alsocalled(properly)agnosticallylearninga1-DGaussian
![Page 28: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/28.jpg)
Fact[Folklore]:Givensamplesfromadistributionthatareε-closeintotalvariationdistancetoa1-DGaussian
themedianandMADrecoverestimatesthatsatisfy
where
Whataboutrobustestimationinhigh-dimensions?
![Page 29: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/29.jpg)
Whataboutrobustestimationinhigh-dimensions?
e.g.microarrayswith10kgenes
Fact[Folklore]:Givensamplesfromadistributionthatareε-closeintotalvariationdistancetoa1-DGaussian
themedianandMADrecoverestimatesthatsatisfy
where
![Page 30: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/30.jpg)
PartI:Introduction
� RobustEstimationinOne-dimension� Robustnessvs.HardnessinHigh-dimensions
� OurResults
PartII:AgnosticallyLearningaGaussian
� ParameterDistance� DetectingWhenanEstimatorisCompromised
� FilteringandConvexProgramming� UnknownCovariance
OUTLINE
PartIII:ExperimentsandExtensions
![Page 31: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/31.jpg)
PartI:Introduction
� RobustEstimationinOne-dimension� Robustnessvs.HardnessinHigh-dimensions
� OurResults
PartII:AgnosticallyLearningaGaussian
� ParameterDistance� DetectingWhenanEstimatorisCompromised
� FilteringandConvexProgramming� UnknownCovariance
OUTLINE
PartIII:ExperimentsandExtensions
![Page 32: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/32.jpg)
MainProblem:Givensamplesfromadistributionthatareε-closeintotalvariationdistancetoad-dimensionalGaussian
giveanefficientalgorithmtofindparametersthatsatisfy
![Page 33: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/33.jpg)
MainProblem:Givensamplesfromadistributionthatareε-closeintotalvariationdistancetoad-dimensionalGaussian
giveanefficientalgorithmtofindparametersthatsatisfy
SpecialCases:
(1)Unknownmean
(2)Unknowncovariance
![Page 34: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/34.jpg)
ACOMPENDIUMOFAPPROACHES
ErrorGuarantee
RunningTime
UnknownMean
![Page 35: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/35.jpg)
ACOMPENDIUMOFAPPROACHES
ErrorGuarantee
RunningTime
TukeyMedian
UnknownMean
![Page 36: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/36.jpg)
ACOMPENDIUMOFAPPROACHES
ErrorGuarantee
RunningTime
TukeyMedian
UnknownMean
O(ε)
![Page 37: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/37.jpg)
ACOMPENDIUMOFAPPROACHES
ErrorGuarantee
RunningTime
TukeyMedian
UnknownMean
O(ε) NP-Hard
![Page 38: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/38.jpg)
ACOMPENDIUMOFAPPROACHES
ErrorGuarantee
RunningTime
TukeyMedian
UnknownMean
O(ε) NP-Hard
GeometricMedian
![Page 39: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/39.jpg)
ACOMPENDIUMOFAPPROACHES
ErrorGuarantee
RunningTime
TukeyMedian
UnknownMean
O(ε) NP-Hard
GeometricMedian poly(d,N)
![Page 40: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/40.jpg)
ACOMPENDIUMOFAPPROACHES
ErrorGuarantee
RunningTime
TukeyMedian
UnknownMean
O(ε) NP-Hard
GeometricMedian poly(d,N)O(ε√d)
![Page 41: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/41.jpg)
ACOMPENDIUMOFAPPROACHES
ErrorGuarantee
RunningTime
TukeyMedian
UnknownMean
O(ε) NP-Hard
GeometricMedian poly(d,N)O(ε√d)
Tournament O(ε) NO(d)
![Page 42: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/42.jpg)
ACOMPENDIUMOFAPPROACHES
ErrorGuarantee
RunningTime
TukeyMedian
UnknownMean
O(ε) NP-Hard
GeometricMedian poly(d,N)O(ε√d)
Tournament O(ε) NO(d)
O(ε√d)Pruning O(dN)
![Page 43: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/43.jpg)
ACOMPENDIUMOFAPPROACHES
ErrorGuarantee
RunningTime
TukeyMedian O(ε) NP-Hard
GeometricMedian O(ε√d) poly(d,N)
Tournament O(ε) NO(d)
O(ε√d)Pruning O(dN)
UnknownMean
…
![Page 44: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/44.jpg)
ThePriceofRobustness?
Allknownestimatorsarehardtocomputeorlosepolynomial factorsinthedimension
![Page 45: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/45.jpg)
ThePriceofRobustness?
Allknownestimatorsarehardtocomputeorlosepolynomial factorsinthedimension
Equivalently:Computationallyefficientestimatorscanonlyhandle
fractionoferrorsandgetnon-trivial(TV<1)guarantees
![Page 46: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/46.jpg)
ThePriceofRobustness?
Allknownestimatorsarehardtocomputeorlosepolynomial factorsinthedimension
Equivalently:Computationallyefficientestimatorscanonlyhandle
fractionoferrorsandgetnon-trivial(TV<1)guarantees
![Page 47: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/47.jpg)
ThePriceofRobustness?
Allknownestimatorsarehardtocomputeorlosepolynomial factorsinthedimension
Equivalently:Computationallyefficientestimatorscanonlyhandle
fractionoferrorsandgetnon-trivial(TV<1)guarantees
Isrobustestimationalgorithmicallypossibleinhigh-dimensions?
![Page 48: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/48.jpg)
PartI:Introduction
� RobustEstimationinOne-dimension� Robustnessvs.HardnessinHigh-dimensions
� OurResults
PartII:AgnosticallyLearningaGaussian
� ParameterDistance� DetectingWhenanEstimatorisCompromised
� FilteringandConvexProgramming� UnknownCovariance
OUTLINE
PartIII:ExperimentsandExtensions
![Page 49: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/49.jpg)
PartI:Introduction
� RobustEstimationinOne-dimension� Robustnessvs.HardnessinHigh-dimensions
� OurResults
PartII:AgnosticallyLearningaGaussian
� ParameterDistance� DetectingWhenanEstimatorisCompromised
� FilteringandConvexProgramming� UnknownCovariance
OUTLINE
PartIII:ExperimentsandExtensions
![Page 50: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/50.jpg)
OURRESULTS
Theorem[Diakonikolas,Li,Kamath,Kane,Moitra,Stewart‘16]:Thereisanalgorithmwhengivensamplesfromadistributionthatisε-closeintotalvariationdistancetoad-dimensionalGaussianfindsparametersthatsatisfy
Robustestimationishigh-dimensionsisalgorithmicallypossible!
Moreoverthealgorithmrunsintimepoly(N,d)
![Page 51: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/51.jpg)
OURRESULTS
Theorem[Diakonikolas,Li,Kamath,Kane,Moitra,Stewart‘16]:Thereisanalgorithmwhengivensamplesfromadistributionthatisε-closeintotalvariationdistancetoad-dimensionalGaussianfindsparametersthatsatisfy
Robustestimationishigh-dimensionsisalgorithmicallypossible!
Moreoverthealgorithmrunsintimepoly(N,d)
Alternatively:CanapproximatetheTukeymedian,etc,ininterestingsemi-randommodels
![Page 52: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/52.jpg)
Simultaneously[Lai,Rao,Vempala ‘16]gaveagnosticalgorithmsthatachieve:
andworkfornon-Gaussiandistributionstoo
![Page 53: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/53.jpg)
Simultaneously[Lai,Rao,Vempala ‘16]gaveagnosticalgorithmsthatachieve:
andworkfornon-Gaussiandistributionstoo
Manyotherapplicationsacrossbothpapers:productdistributions,mixturesofsphericalGaussians,SVD,ICA
![Page 54: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/54.jpg)
AGENERALRECIPE
Robustestimationinhigh-dimensions:
� Step#1:Findanappropriateparameterdistance
� Step#2:Detectwhenthenaïveestimatorhasbeencompromised
� Step#3:Findgoodparameters,ormakeprogressFiltering:FastandpracticalConvexProgramming:Bettersamplecomplexity
![Page 55: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/55.jpg)
AGENERALRECIPE
Robustestimationinhigh-dimensions:
� Step#1:Findanappropriateparameterdistance
� Step#2:Detectwhenthenaïveestimatorhasbeencompromised
� Step#3:Findgoodparameters,ormakeprogressFiltering:FastandpracticalConvexProgramming:Bettersamplecomplexity
Let’sseehowthisworksforunknownmean…
![Page 56: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/56.jpg)
PartI:Introduction
� RobustEstimationinOne-dimension� Robustnessvs.HardnessinHigh-dimensions
� OurResults
PartII:AgnosticallyLearningaGaussian
� ParameterDistance� DetectingWhenanEstimatorisCompromised
� FilteringandConvexProgramming� UnknownCovariance
OUTLINE
PartIII:ExperimentsandExtensions
![Page 57: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/57.jpg)
PartI:Introduction
� RobustEstimationinOne-dimension� Robustnessvs.HardnessinHigh-dimensions
� OurResults
PartII:AgnosticallyLearningaGaussian
� ParameterDistance� DetectingWhenanEstimatorisCompromised
� FilteringandConvexProgramming� UnknownCovariance
OUTLINE
PartIII:ExperimentsandExtensions
![Page 58: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/58.jpg)
PARAMETERDISTANCE
Step#1:FindanappropriateparameterdistanceforGaussians
![Page 59: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/59.jpg)
PARAMETERDISTANCE
Step#1:FindanappropriateparameterdistanceforGaussians
ABasicFact:
(1)
![Page 60: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/60.jpg)
PARAMETERDISTANCE
Step#1:FindanappropriateparameterdistanceforGaussians
ABasicFact:
(1)
ThiscanbeprovenusingPinsker’s Inequality
andthewell-knownformulaforKL-divergencebetweenGaussians
![Page 61: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/61.jpg)
PARAMETERDISTANCE
Step#1:FindanappropriateparameterdistanceforGaussians
ABasicFact:
(1)
![Page 62: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/62.jpg)
PARAMETERDISTANCE
Step#1:FindanappropriateparameterdistanceforGaussians
ABasicFact:
(1)
Corollary:Ifourestimate(intheunknownmeancase)satisfies
then
![Page 63: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/63.jpg)
PARAMETERDISTANCE
Step#1:FindanappropriateparameterdistanceforGaussians
ABasicFact:
(1)
Corollary:Ifourestimate(intheunknownmeancase)satisfies
then
OurnewgoalistobecloseinEuclideandistance
![Page 64: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/64.jpg)
PartI:Introduction
� RobustEstimationinOne-dimension� Robustnessvs.HardnessinHigh-dimensions
� OurResults
PartII:AgnosticallyLearningaGaussian
� ParameterDistance� DetectingWhenanEstimatorisCompromised
� FilteringandConvexProgramming� UnknownCovariance
OUTLINE
PartIII:ExperimentsandExtensions
![Page 65: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/65.jpg)
PartI:Introduction
� RobustEstimationinOne-dimension� Robustnessvs.HardnessinHigh-dimensions
� OurResults
PartII:AgnosticallyLearningaGaussian
� ParameterDistance� DetectingWhenanEstimatorisCompromised
� FilteringandConvexProgramming� UnknownCovariance
OUTLINE
PartIII:ExperimentsandExtensions
![Page 66: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/66.jpg)
DETECTINGCORRUPTIONS
Step#2:Detectwhenthenaïveestimatorhasbeencompromised
![Page 67: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/67.jpg)
DETECTINGCORRUPTIONS
Step#2:Detectwhenthenaïveestimatorhasbeencompromised
=uncorrupted=corrupted
![Page 68: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/68.jpg)
DETECTINGCORRUPTIONS
Step#2:Detectwhenthenaïveestimatorhasbeencompromised
=uncorrupted=corrupted
Thereisadirectionoflarge(>1)variance
![Page 69: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/69.jpg)
KeyLemma:IfX1,X2,…XN comefromadistributionthatisε-closetoandthenfor
(1) (2)
withprobabilityatleast1-δ
![Page 70: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/70.jpg)
KeyLemma:IfX1,X2,…XN comefromadistributionthatisε-closetoandthenfor
(1) (2)
withprobabilityatleast1-δ
Take-away:Anadversaryneedstomessupthesecondmomentinordertocorruptthefirstmoment
![Page 71: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/71.jpg)
PartI:Introduction
� RobustEstimationinOne-dimension� Robustnessvs.HardnessinHigh-dimensions
� OurResults
PartII:AgnosticallyLearningaGaussian
� ParameterDistance� DetectingWhenanEstimatorisCompromised
� FilteringandConvexProgramming� UnknownCovariance
OUTLINE
PartIII:ExperimentsandExtensions
![Page 72: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/72.jpg)
PartI:Introduction
� RobustEstimationinOne-dimension� Robustnessvs.HardnessinHigh-dimensions
� OurResults
PartII:AgnosticallyLearningaGaussian
� ParameterDistance� DetectingWhenanEstimatorisCompromised
� FilteringandConvexProgramming� UnknownCovariance
OUTLINE
PartIII:ExperimentsandExtensions
![Page 73: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/73.jpg)
OURALGORITHM(S)
Step#3:Eitherfindgoodparameters,orremovemanyoutliers
![Page 74: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/74.jpg)
OURALGORITHM(S)
Step#3:Eitherfindgoodparameters,orremovemanyoutliers
FilteringApproach:Supposethat:
![Page 75: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/75.jpg)
OURALGORITHM(S)
Step#3:Eitherfindgoodparameters,orremovemanyoutliers
FilteringApproach:Supposethat:
Wecanthrowoutmorecorruptedthanuncorruptedpoints:
v
wherevisthedirectionoflargestvariance
![Page 76: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/76.jpg)
OURALGORITHM(S)
Step#3:Eitherfindgoodparameters,orremovemanyoutliers
FilteringApproach:Supposethat:
Wecanthrowoutmorecorruptedthanuncorruptedpoints:
v
wherevisthedirectionoflargestvariance,andThasaformula
![Page 77: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/77.jpg)
OURALGORITHM(S)
Step#3:Eitherfindgoodparameters,orremovemanyoutliers
FilteringApproach:Supposethat:
Wecanthrowoutmorecorruptedthanuncorruptedpoints:
v
T
wherevisthedirectionoflargestvariance,andThasaformula
![Page 78: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/78.jpg)
OURALGORITHM(S)
Step#3:Eitherfindgoodparameters,orremovemanyoutliers
FilteringApproach:Supposethat:
Wecanthrowoutmorecorruptedthanuncorruptedpoints
![Page 79: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/79.jpg)
OURALGORITHM(S)
Step#3:Eitherfindgoodparameters,orremovemanyoutliers
FilteringApproach:Supposethat:
Wecanthrowoutmorecorruptedthanuncorruptedpoints
Ifwecontinuetoolong,we’dhavenocorruptedpointsleft!
![Page 80: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/80.jpg)
OURALGORITHM(S)
Step#3:Eitherfindgoodparameters,orremovemanyoutliers
FilteringApproach:Supposethat:
Wecanthrowoutmorecorruptedthanuncorruptedpoints
Ifwecontinuetoolong,we’dhavenocorruptedpointsleft!
Eventuallywefind(certifiably)goodparameters
![Page 81: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/81.jpg)
OURALGORITHM(S)
Step#3:Eitherfindgoodparameters,orremovemanyoutliers
FilteringApproach:Supposethat:
Wecanthrowoutmorecorruptedthanuncorruptedpoints
Ifwecontinuetoolong,we’dhavenocorruptedpointsleft!
Eventuallywefind(certifiably)goodparameters
RunningTime: SampleComplexity:
![Page 82: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/82.jpg)
OURALGORITHM(S)
Step#3:Eitherfindgoodparameters,orremovemanyoutliers
FilteringApproach:Supposethat:
Wecanthrowoutmorecorruptedthanuncorruptedpoints
Ifwecontinuetoolong,we’dhavenocorruptedpointsleft!
Eventuallywefind(certifiably)goodparameters
RunningTime: SampleComplexity:ConcentrationofLTFs
![Page 83: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/83.jpg)
PartI:Introduction
� RobustEstimationinOne-dimension� Robustnessvs.HardnessinHigh-dimensions
� OurResults
PartII:AgnosticallyLearningaGaussian
� ParameterDistance� DetectingWhenanEstimatorisCompromised
� FilteringandConvexProgramming� UnknownCovariance
OUTLINE
PartIII:ExperimentsandExtensions
![Page 84: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/84.jpg)
PartI:Introduction
� RobustEstimationinOne-dimension� Robustnessvs.HardnessinHigh-dimensions
� OurResults
PartII:AgnosticallyLearningaGaussian
� ParameterDistance� DetectingWhenanEstimatorisCompromised
� FilteringandConvexProgramming� UnknownCovariance
OUTLINE
PartIII:ExperimentsandExtensions
![Page 85: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/85.jpg)
AGENERALRECIPE
Robustestimationinhigh-dimensions:
� Step#1:Findanappropriateparameterdistance
� Step#2:Detectwhenthenaïveestimatorhasbeencompromised
� Step#3:Findgoodparameters,ormakeprogressFiltering:FastandpracticalConvexProgramming:Bettersamplecomplexity
![Page 86: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/86.jpg)
AGENERALRECIPE
Robustestimationinhigh-dimensions:
� Step#1:Findanappropriateparameterdistance
� Step#2:Detectwhenthenaïveestimatorhasbeencompromised
� Step#3:Findgoodparameters,ormakeprogressFiltering:FastandpracticalConvexProgramming:Bettersamplecomplexity
Howaboutforunknowncovariance?
![Page 87: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/87.jpg)
PARAMETERDISTANCE
Step#1:FindanappropriateparameterdistanceforGaussians
![Page 88: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/88.jpg)
PARAMETERDISTANCE
Step#1:FindanappropriateparameterdistanceforGaussians
AnotherBasicFact:
(2)
![Page 89: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/89.jpg)
PARAMETERDISTANCE
Step#1:FindanappropriateparameterdistanceforGaussians
AnotherBasicFact:
Again,provenusingPinsker’s Inequality
(2)
![Page 90: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/90.jpg)
PARAMETERDISTANCE
Step#1:FindanappropriateparameterdistanceforGaussians
AnotherBasicFact:
Again,provenusingPinsker’s Inequality
(2)
Ournewgoalistofindanestimatethatsatisfies:
![Page 91: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/91.jpg)
PARAMETERDISTANCE
Step#1:FindanappropriateparameterdistanceforGaussians
AnotherBasicFact:
Again,provenusingPinsker’s Inequality
(2)
Ournewgoalistofindanestimatethatsatisfies:
Distanceseemsstrange,butit’stherightonetousetoboundTV
![Page 92: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/92.jpg)
UNKNOWNCOVARIANCE
Whatifwearegivensamplesfrom?
![Page 93: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/93.jpg)
UNKNOWNCOVARIANCE
Whatifwearegivensamplesfrom?
Howdowedetectifthenaïveestimatoriscompromised?
![Page 94: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/94.jpg)
UNKNOWNCOVARIANCE
Whatifwearegivensamplesfrom?
Howdowedetectifthenaïveestimatoriscompromised?
KeyFact:Let and
Thenrestrictedtoflattenings ofdxdsymmetricmatrices
![Page 95: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/95.jpg)
UNKNOWNCOVARIANCE
Whatifwearegivensamplesfrom?
Howdowedetectifthenaïveestimatoriscompromised?
KeyFact:Let and
Thenrestrictedtoflattenings ofdxdsymmetricmatrices
ProofusesIsserlis’s Theorem
![Page 96: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/96.jpg)
UNKNOWNCOVARIANCE
needtoprojectout
Whatifwearegivensamplesfrom?
Howdowedetectifthenaïveestimatoriscompromised?
KeyFact:Let and
Thenrestrictedtoflattenings ofdxdsymmetricmatrices
![Page 97: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/97.jpg)
KeyIdea: Transformthedata,lookforrestrictedlargeeigenvalues
![Page 98: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/98.jpg)
KeyIdea: Transformthedata,lookforrestrictedlargeeigenvalues
![Page 99: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/99.jpg)
KeyIdea: Transformthedata,lookforrestrictedlargeeigenvalues
Ifwerethetruecovariance,wewouldhaveforinliers
![Page 100: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/100.jpg)
KeyIdea: Transformthedata,lookforrestrictedlargeeigenvalues
Ifwerethetruecovariance,wewouldhaveforinliers,inwhichcase:
wouldhavesmallrestrictedeigenvalues
![Page 101: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/101.jpg)
KeyIdea: Transformthedata,lookforrestrictedlargeeigenvalues
Ifwerethetruecovariance,wewouldhaveforinliers,inwhichcase:
wouldhavesmallrestrictedeigenvalues
Take-away:Anadversaryneedstomessupthe(restricted)fourthmomentinordertocorruptthesecondmoment
![Page 102: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/102.jpg)
ASSEMBLINGTHEALGORITHM
Givensamplesthatareε-closeintotalvariationdistancetoad-dimensionalGaussian
![Page 103: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/103.jpg)
ASSEMBLINGTHEALGORITHM
Givensamplesthatareε-closeintotalvariationdistancetoad-dimensionalGaussian
Step#1:Doublingtrick
![Page 104: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/104.jpg)
ASSEMBLINGTHEALGORITHM
Givensamplesthatareε-closeintotalvariationdistancetoad-dimensionalGaussian
Step#1:Doublingtrick
Nowusealgorithmforunknowncovariance
![Page 105: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/105.jpg)
ASSEMBLINGTHEALGORITHM
Givensamplesthatareε-closeintotalvariationdistancetoad-dimensionalGaussian
Step#1:Doublingtrick
Nowusealgorithmforunknowncovariance
Step#2:(Agnostic)isotropicposition
![Page 106: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/106.jpg)
ASSEMBLINGTHEALGORITHM
Givensamplesthatareε-closeintotalvariationdistancetoad-dimensionalGaussian
Step#1:Doublingtrick
Nowusealgorithmforunknowncovariance
Step#2:(Agnostic)isotropicposition
rightdistance,ingeneralcase
![Page 107: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/107.jpg)
ASSEMBLINGTHEALGORITHM
Givensamplesthatareε-closeintotalvariationdistancetoad-dimensionalGaussian
Step#1:Doublingtrick
Nowusealgorithmforunknowncovariance
Step#2:(Agnostic)isotropicposition
Nowusealgorithmforunknownmeanrightdistance,ingeneralcase
![Page 108: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/108.jpg)
PartI:Introduction
� RobustEstimationinOne-dimension� Robustnessvs.HardnessinHigh-dimensions
� OurResults
PartII:AgnosticallyLearningaGaussian
� ParameterDistance� DetectingWhenanEstimatorisCompromised
� FilteringandConvexProgramming� UnknownCovariance
OUTLINE
PartIII:ExperimentsandExtensions
![Page 109: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/109.jpg)
PartI:Introduction
� RobustEstimationinOne-dimension� Robustnessvs.HardnessinHigh-dimensions
� OurResults
PartII:AgnosticallyLearningaGaussian
� ParameterDistance� DetectingWhenanEstimatorisCompromised
� FilteringandConvexProgramming� UnknownCovariance
OUTLINE
PartIII:ExperimentsandExtensions
![Page 110: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/110.jpg)
FURTHERRESULTS
Userestrictedeigenvalueproblemstodetectoutliers
![Page 111: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/111.jpg)
FURTHERRESULTS
Userestrictedeigenvalueproblemstodetectoutliers
BinaryProductDistributions:
![Page 112: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/112.jpg)
FURTHERRESULTS
Userestrictedeigenvalueproblemstodetectoutliers
BinaryProductDistributions:
MixturesofTwoc-BalancedBinaryProductDistributions:
![Page 113: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/113.jpg)
FURTHERRESULTS
Userestrictedeigenvalueproblemstodetectoutliers
BinaryProductDistributions:
MixturesofTwoc-BalancedBinaryProductDistributions:
MixturesofkSphericalGaussians:
![Page 114: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/114.jpg)
SYNTHETICEXPERIMENTS
Errorratesonsyntheticdata(unknownmean):
+10%noise
![Page 115: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/115.jpg)
SYNTHETICEXPERIMENTS
Errorratesonsyntheticdata(unknownmean):
100 200 300 400
0
0.5
1
1.5
dimension
excess` 2
error
Filtering
LRVMean
Sample mean w/ noise
Pruning
RANSAC Geometric Median
100 200 300 400
0.04
0.06
0.08
0.1
0.12
0.14
dimension
excess` 2
error
![Page 116: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/116.jpg)
SYNTHETICEXPERIMENTS
Errorratesonsyntheticdata(unknowncovariance,isotropic):
+10%noise
closetoidentity
![Page 117: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/117.jpg)
SYNTHETICEXPERIMENTS
20 40 60 80 100
0
0.5
1
1.5
dimension
excess` 2
error
Filtering
LRVCov
Sample covariance w/ noise
Pruning
RANSAC
20 40 60 80 100
0
0.1
0.2
0.3
0.4
dimension
excess` 2
error
Errorratesonsyntheticdata(unknowncovariance,isotropic):
![Page 118: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/118.jpg)
SYNTHETICEXPERIMENTS
Errorratesonsyntheticdata(unknowncovariance,anisotropic):
+10%noise
farfromidentity
![Page 119: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/119.jpg)
SYNTHETICEXPERIMENTS
20 40 60 80 100
0
50
100
150
200
dimension
excess` 2
error
Filtering
LRVCov
Sample covariance w/ noise
Pruning
RANSAC
20 40 60 80 100
0
0.5
1
dimension
excess` 2
error
Errorratesonsyntheticdata(unknowncovariance,anisotropic):
![Page 120: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/120.jpg)
REALDATAEXPERIMENTS
Famousstudyof[Novembre etal.‘08]:TaketoptwosingularvectorsofpeoplexSNPmatrix(POPRES)
![Page 121: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/121.jpg)
REALDATAEXPERIMENTS
Famousstudyof[Novembre etal.‘08]:TaketoptwosingularvectorsofpeoplexSNPmatrix(POPRES)
-0.2
-0.1
0
0.1
0.2
0.3
-0.15 -0.1 -0.05 0 0.05 0.1 0.15 0.2
Original Data
![Page 122: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/122.jpg)
REALDATAEXPERIMENTS
Famousstudyof[Novembre etal.‘08]:TaketoptwosingularvectorsofpeoplexSNPmatrix(POPRES)
-0.2
-0.1
0
0.1
0.2
0.3
-0.15 -0.1 -0.05 0 0.05 0.1 0.15 0.2
Original Data
![Page 123: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/123.jpg)
REALDATAEXPERIMENTS
Famousstudyof[Novembre etal.‘08]:TaketoptwosingularvectorsofpeoplexSNPmatrix(POPRES)
-0.2
-0.1
0
0.1
0.2
0.3
-0.15 -0.1 -0.05 0 0.05 0.1 0.15 0.2
Original Data
“GenesMirrorGeographyinEurope”
![Page 124: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/124.jpg)
REALDATAEXPERIMENTS
Canwefindsuchpatternsinthepresenceofnoise?
![Page 125: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/125.jpg)
REALDATAEXPERIMENTS
Canwefindsuchpatternsinthepresenceofnoise?
-0.2 -0.1 0 0.1 0.2 0.3-0.15
-0.1
-0.05
0
0.05
0.1
0.15
0.2Pruning Projection
10%noise
WhatPCAfinds
![Page 126: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/126.jpg)
REALDATAEXPERIMENTS
Canwefindsuchpatternsinthepresenceofnoise?
-0.2 -0.1 0 0.1 0.2 0.3-0.15
-0.1
-0.05
0
0.05
0.1
0.15
0.2Pruning Projection
10%noise
WhatPCAfinds
![Page 127: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/127.jpg)
-0.2 -0.1 0 0.1 0.2 0.3
-0.15
-0.1
-0.05
0
0.05
0.1
0.15
0.2RANSAC Projection
REALDATAEXPERIMENTS
Canwefindsuchpatternsinthepresenceofnoise?
10%noise
WhatRANSACfinds
![Page 128: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/128.jpg)
-0.2
-0.1
0
0.1
0.2
0.3
-0.15 -0.1 -0.05 0 0.05 0.1 0.15 0.2
XCS Projection
REALDATAEXPERIMENTS
Canwefindsuchpatternsinthepresenceofnoise?
10%noise
WhatrobustPCA(viaSDPs)finds
![Page 129: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/129.jpg)
-0.2
-0.1
0
0.1
0.2
0.3
-0.15-0.1-0.0500.050.10.150.2
Filter Projection
REALDATAEXPERIMENTS
Canwefindsuchpatternsinthepresenceofnoise?
10%noise
Whatourmethodsfind
![Page 130: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/130.jpg)
-0.2
-0.1
0
0.1
0.2
0.3
-0.15-0.1-0.0500.050.10.150.2
Filter Projection
-0.2
-0.1
0
0.1
0.2
0.3
-0.15 -0.1 -0.05 0 0.05 0.1 0.15 0.2
Original Data
REALDATAEXPERIMENTS
10%noise
Whatourmethodsfind
nonoise
Thepowerofprovablyrobustestimation:
![Page 131: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/131.jpg)
LOOKINGFORWARD
CanalgorithmsforagnosticallylearningaGaussianhelpinexploratorydataanalysisinhigh-dimensions?
![Page 132: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/132.jpg)
LOOKINGFORWARD
CanalgorithmsforagnosticallylearningaGaussianhelpinexploratorydataanalysisinhigh-dimensions?
Isn’tthiswhatwewouldhavebeendoingwithrobuststatisticalestimators,ifwehadthemallalong?
![Page 133: Robust Statistics, Revisited - People | MIT CSAILpeople.csail.mit.edu/moitra/docs/robust2.pdf · 2017. 3. 10. · Robust estimation is high-dimensions is algorithmically possible!](https://reader033.vdocuments.mx/reader033/viewer/2022052816/60abe8d98c2b8509162d2b78/html5/thumbnails/133.jpg)
Thanks!AnyQuestions?
Summary:� Nearlyoptimalalgorithmforagnosticallylearningahigh-dimensionalGaussian
� Generalrecipeusingrestrictedeigenvalueproblems� Furtherapplicationstoothermixturemodels� Ispractical,robuststatisticswithinreach?