vulnerability disclosure in the age of social media...
TRANSCRIPT
12/7/15
1
VulnerabilityDisclosureintheAgeofSocialMedia:Exploi<ngTwi?erforPredic<ngReal-WorldExploits
TudorDumitrașAssistantProfessor,ECE&MC2UniversityofMaryland,CollegePark
WorkwithCarlSabo?keandOctavianSuciu
SomethingAboutMeFirst
TudorDumitrașWeb:www.umiacs.umd.edu/~tdumitraEmail:[email protected]• Ph.D.,CarnegieMellonUniversity• 2.5yr.atSymantecResearchLabs• JoinedUMDin2013
TudorDumitraș::VulnerabilityDisclosureintheAgeofSocialMedia 2
12/7/15
2
MoreandmoresoLwarevulnerabili<esarediscovered…
2014CVEIDsformatchange:nolongerlimitedto9,999vulnerabiliTes/year
Heartbleed
Poodle
Shellshock
TudorDumitraș::VulnerabilityDisclosureintheAgeofSocialMedia 3
hVp://blog.osvdb.org/2015/02/02/vendors-sure-like-to-wave-the-coordinaTon-flag-revisiTng-the-perfect-storm/
12/7/15
3
ResearchQues<ons
• Howtopriori<zetheresponsetovulnerabilitydisclosures?• CanforecastvulnerabiliTesexploitedinthewild?– …earlierthanexisTngdatasources?– …andwithfewerfalseposi<ves?
Ourapproach:Twi?eranaly<cs
5
Hackers
Systemadministrators
Vulnerabilityresearchers
TudorDumitraș::VulnerabilityDisclosureintheAgeofSocialMedia
Demonstra<on
TudorDumitraș::VulnerabilityDisclosureintheAgeofSocialMedia 6
hVp://ter.ps/sec15demo
12/7/15
4
SystemDesign[USENIXSecurity’15]
7
GroundTruth
Features
Predic<ons
LinearSVM
MachineLearning
Precision:oneorderofmagnitudebeVerthanCVSS
DetecTon:medianof2daysaheadofexisTngdatasets
TudorDumitraș::VulnerabilityDisclosureintheAgeofSocialMedia
AdversarialInterference
• TwiVerisfreeandopentoallusers• CouldanadversarypostfalseinformaToninordertotrickadetector?
8TudorDumitraș::VulnerabilityDisclosureintheAgeofSocialMedia
12/7/15
5
TalkOutline
• DesignandimplementaTonofatechniqueforearlyexploitdetecTonusingsocialmedia
• PerformanceevaluaTonfordetecTngexploitsfoundinthewild
• Analysisofsystemrobustnesstoadversarialinterference
• SecurityimplicaTons
9TudorDumitraș::VulnerabilityDisclosureintheAgeofSocialMedia
TalkOutline
• Designandimplementa<onofatechniqueforearlyexploitdetec<onusingsocialmedia
• PerformanceevaluaTonfordetecTngexploitsfoundinthewild
• Analysisofsystemrobustnesstoadversarialinterference
• SecurityimplicaTons
10TudorDumitraș::VulnerabilityDisclosureintheAgeofSocialMedia
12/7/15
6
Twi?erDataset
• TwiVerPublicStream– February2014-January2015– 1.1billiontweets
• TrackingtheCVEkeyword• Collectedunsampledcorpus– 287,717tweets– 5,865vulnerabiliTes
11TudorDumitraș::VulnerabilityDisclosureintheAgeofSocialMedia
SystemDesign
12
Ground "Truth"
Features"
Classification"Results"
Testing"
Linear SVM"Training"
Database Features!
OSVDBCategory
#ofReferencesinNVD
…!
Twitter Traffic Features!
NumberofTweets
NumberofRetweets
…!
CVSS v2 Base Metric!
AvailabilityImpact
AuthenTcaTon
…!
Twitter Word Features!
exploit
web
…!
TudorDumitraș::VulnerabilityDisclosureintheAgeofSocialMedia
12/7/15
7
Detec<ngExploitsintheWild
Classification"Results"
Testing"
Linear SVM"Training"
Ground "Truth"
Features"
TudorDumitraș::VulnerabilityDisclosureintheAgeofSocialMedia 13
ClassifierEvalua<on
• Aclassifiercanmaketwokindsoferrors– FalsePosi<ve=markedasexploitedbutnotexploitedinthewild
– FalseNega<ve=notmarkedasexploited,butexploitedinthewild
• Precision=fracTonofvulnerabiliTesmarkedasexploitedthatareactuallyexploited– FalseposiTveshurtprecision
• Recall=fracTonofexploitedvulnerabiliTesthataremarkedasexploited
TudorDumitraș::VulnerabilityDisclosureintheAgeofSocialMedia 14
12/7/15
8
Precision
}<9%
BaselineClassifier• UsingCVSSScoreasindicatorofanexploit• CVSSmarksmanyvulnerabiliTesasexploitable
15TudorDumitraș::VulnerabilityDisclosureintheAgeofSocialMedia
Detec<ngExploitsintheWild
• CVSSScore:verylowprecision,highrecall
16TudorDumitraș::VulnerabilityDisclosureintheAgeofSocialMedia
12/7/15
9
Detec<ngExploitsintheWild
• DatabaseInformaTon:Highrecall,lowprecision
17TudorDumitraș::VulnerabilityDisclosureintheAgeofSocialMedia
Detec<ngExploitsintheWild
• TwiVerWordfeatures:lowrecall,highprecision
18TudorDumitraș::VulnerabilityDisclosureintheAgeofSocialMedia
12/7/15
10
Detec<ngExploitsintheWild
• TwiVerTrafficfeatures:higherrecall,lowerprecision
19TudorDumitraș::VulnerabilityDisclosureintheAgeofSocialMedia
Detec<ngExploitsintheWild
• Combiningallfeatures:variableregularizaTonresultsinaprecision/recalltradeoff
20TudorDumitraș::VulnerabilityDisclosureintheAgeofSocialMedia
12/7/15
11
ImprovingthePerformance
• Filteringbasedongroundtruthcoverageandtweetvolume
21TudorDumitraș::MachineLearningTechniquesforPrevenTngtheGlobalMalwareDisseminaTon
EarlyPredic<onofExploits
22
Streaming
GroundTruth
Features
Training
LinearSVM
PredicTonThreshold
Date
TudorDumitraș::VulnerabilityDisclosureintheAgeofSocialMedia
12/7/15
12
DetecTonmostlyoccursearlierthanthesignatures
TweetsBeforeSignatures
• TradeoffbetweenprecisionanddetecTonleadTme– MediandetecTon:2daysaheadofSymantecsignatures– 45%classificaTonprecision
23TudorDumitraș::VulnerabilityDisclosureintheAgeofSocialMedia
AdversarialInterference
24
ClassificaTonResults
TesTng
LinearSVMTraining
GroundTruth
Features
ExploratoryAVack
CausaTveAVack
TudorDumitraș::VulnerabilityDisclosureintheAgeofSocialMedia
12/7/15
13
A?acksAgainsttheExploitDetector
• Canwepreventtheadversaryfrompoisoningthetrainingdataset?– No.TwiVerisafreeandopenservice.
• Canwekeepthefeaturessecret?– No.Ourgroundtruthcomesfrompublicsources.
• Istheadversaryresourcebound?– Yes.AdversarymustcontroltheproperTesofmulTpleaccounts.
25TudorDumitraș::VulnerabilityDisclosureintheAgeofSocialMedia
AdversaryModel
• Adversary’sgoal:tointroducefalseposi<ves
• SimulaTonofcausa<vea?acks– 3adversarytypes
• AdversariescannotpreventbenignusersfromposTng
26TudorDumitraș::VulnerabilityDisclosureintheAgeofSocialMedia
12/7/15
14
BlabberingAdversary
• Randomlypoststweets,noknowledgeaboutfeatures
27
Randomnoiseaffectsthesystemminimally
TudorDumitraș::VulnerabilityDisclosureintheAgeofSocialMedia
WordCopycatAdversary
• MirrorsthestaTsTcsofwordscorrespondingtoexploitedvulnerabiliTes
28
Damageisboundduetootherfeatures(e.g.Traffic,CVSS,Databases)
TudorDumitraș::VulnerabilityDisclosureintheAgeofSocialMedia
12/7/15
15
FullCopycatAdversary• Sybil-like:controlsmulTpleaccounts– ManipulatesallTwiVerfeaturesexceptaccountcreaTondateandaccountverificaTon
29
Damageisboundonlybynon-TwiVerfeatures
Forresilience,needlistoftrustedusersMostinformaTvetweetscomefrom~4,000users
TudorDumitraș::VulnerabilityDisclosureintheAgeofSocialMedia
TalkOutline
• DesignandimplementaTonofatechniqueforearlyexploitdetecTonusingsocialmedia
• PerformanceevaluaTonfordetecTngexploitsfoundinthewild
• Analysisofsystemrobustnesstoadversarialinterference
• Securityimplica<ons
30TudorDumitraș::VulnerabilityDisclosureintheAgeofSocialMedia
12/7/15
16
SecurityImplica<ons• FighTngexploitswithmachinelearning– Canforecastsomevulnerabilityexploits
– Highprecisionandrecallforproblemsthatalreadyhavegoodpredictors(e.g.MSexploitabilityindex)
– Challenges:conceptdriL,adversarialinterference– ModelshavemorepotenTalapplicaTons(e.g.cyberinsurance)
• FewvulnerabiliTesareexploitedinthewild– Exploitscarcityfeltintheundergroundeconomy• Blackholeexploitkit(2013):$100,000budgetforpurchasing0-dayexploits• 0-dayexploitforCVE-2013-3906:bothtargetedaVacksandbotnet-basedmalware
– Challenge:Poorgroundtruthcoverage• BeVerinformaTonsharingwouldimprovethedetectors
TudorDumitraș::MachineLearningTechniquesforPrevenTngtheGlobalMalwareDisseminaTon 31
ThingsIHaven’tToldYouAbout• Miningdownloadergraphstodetectmalware[CCS’15]
• Howwemeasuredthepatchingrateof1,593vulnerabiliTes[Oakland’15]
• HowwemeasuredtheduraTonanprevalenceofzero-dayaVacks[CCS’12]
• CerTficatereissuesandrevocaTonsinthewakeofHeartbleed[IMC’14]
• Securitymetricsbasedonfielddata[RAID’14]
TudorDumitraș::MachineLearningTechniquesforPrevenTngtheGlobalMalwareDisseminaTon 32
12/7/15
17
Students
TudorDumitraș::MachineLearningTechniquesforPrevenTngtheGlobalMalwareDisseminaTon 33
OctavianSuciu
ZiyunZhu
BumJunKwon
YantaoZhang
Collaborators:LeylaBilge,PetrosEfstathopoulos,DanielMarino(SymantecResearchLabs),JiyongJang(IBMResearch),AaronSchulman(Google),JuanCaballero(IMDEASoyware),PoloChau(GeorgiaTech),DavidChoffnes,AlanMislove,ChristoWilson(NortheasternUniversity),AmolDeshpande,DavidLevin,V.S.Subrahmanian(UniversityofMaryland),ChristosFaloutsos(CarnegieMellonUniversity),IulianNeamTu(NJIT),AdityaPrakash,EliTilevich(VirginiaTech)
Thankyou!
TudorDumitraș::VulnerabilityDisclosureintheAgeofSocialMedia
TudorDumitraș[email protected]://www.umiacs.umd.edu/~tdumitra@tudor_dumitras
34
Paperanddetailedfeaturelist: http://ter.ps/sec15exploitDemo: http://ter.ps/sec15demo