stephan neuhaus thomas zimmermann andreas zeller
Post on 02-Jan-2017
240 Views
Preview:
TRANSCRIPT
Predicting Vulnerable Software Components
Stephan Neuhaus
Thomas ZimmermannAndreas Zeller
Security Advisory 2005-12
Title: Livefeed bookmarks can steal cookiesImpact: HighProducts: FirefoxDescription: Earlier versions of Firefox allowed javascript: and data: URLs as Livefeed bookmarks. When they updated the URL would be run in the context of the current page and could be used to steal cookies or data displayed on the page. If the user were on a page with elevated privileges (for example, about:config) when the Livefeed was updated, the feed URL could potentially run arbitrary code on the user's machine.
Security Advisory 2005-13
Title: Window Injection SpoofingSeverity: LowProducts: Firefox, Mozilla SuiteDescription: A website can inject content into a popup opened by another site if the target name of the popup window is known. An attacker who knows you are going to visit that other site could spoof the contents of the popup.
Security Advisory 2005-14
Title: SSL "secure site" indicator spoofingSeverity: ModerateProducts: Firefox, Mozilla SuiteDescription: Various schemes were reported that could cause the "secure site" lock icon to appear and show certificate details for the wrong site. These could be used by phishers to make their spoofs look more legitimate, particularly in windows that hide the address bar showing the true location.
Security Advisory 2005-15Title: Heap overflow possible in UTF8 to Unicode conversionSeverity: HighProducts: Firefox, Thunderbird, Mozilla SuiteDescription: It is possible for a UTF8 string with invalid sequences to trigger a heap overflow of converted Unicode data. Exploitability would depend on the attackers ability to get the string into the buggy converter. General web content is converted elsewhere but we can't rule out the possibility of a successful attack.
Security Advisory 2005-16Title: Spoofing download and security dialogs with overlapping windowsSeverity: HighProducts: Firefox, Mozilla SuiteDescription: Michael Krax demonstrates that the download dialog and security dialogs can be spoofed by partially covering them with an overlapping window. Some users may not notice the OS window border and browser statusbar bisecting what appears to be a single dialog, and be convinced by the spoofing text of the top-most window to click on the "Allow" or "Open" button of the window below.
Vulnerabilities0
Security Advisory 2005-41Title: Privilege escalation via DOM property overridesSeverity: CriticalProducts: Firefox, Mozilla SuiteDescription: moz_bug_r_a4 reported several exploits giving an attacker the ability to install malicious code or steal data, requiring only that the user do commonplace actions like click on a link or open the context menu. The common cause in each case was privileged UI code ("chrome") being overly trusting of DOM nodes from the content window.
Security Advisory 2006-76Title: XSS using outer window's Function objectImpact: HighProducts: Firefox 2.0Description: moz_bug_r_a4 demonstrated that the Function prototype regression described in bug 355161 could be exploited to bypass the protections against cross site script (XSS) injection, which could be used to steal credentials or sensitive data from arbitrary sites or perform destructive actions on behalf of a logged-in user.
Is this new component likely to be vulnerable?What other components are vulnerable?
Vulnerability Database
Version Archive
CodeCodeCodeCode
Component Component Component
Code
PredictorVulture
Code
Programmer Code Complexity
Language
Look for features that areinvariant under evolution
GUI Database Certificates OS
Imports
nsIContent.h
nsIContentUtils.h
nsIScriptSecurityManager.h
✘
✘
✘
✘
✘
✘
✘
✘
✘
✘
✘
✘
✘
✘
✘
✘
✘
✘
✘
✘
✘✔
nsIPrivateDOMEvent.h
nsReadableUtils.h
✘
✘
✘
✘
✘
✘
✘
✘
✘
✘
✘
✘
✘
✘
✘
✘
✘
✘✘
Research Questions
• How well do imports predict vulnerabilities?
• Can imports be used for classification (vulnerable or not) and for regression (number of vulnerabilities)?
Case Study: Mozilla
• CVS from January 4, 2007
• 14,368 C/C++ files
• 134 Security Advisories since January 2005
• Only 424 vulnerable components (4.05%)
⇒ Prediction is challenging
components in Mozilla
vulnerable components424
10,452
4.05%
Mozilla Vulnerabilities
security
nss
lib
libpkix
pkix_pl_nss
modu pki sy
pkix
top uti r
incl
ckfw
builtins ca
ns
freebl
mpi ecl
softoken
ssl
pk11wr
util certd smim
crmf
pki1
pki
pkcs12
certhig
pkcs de
jar cry
bas asn
cmd
zlib lib m pk si fips
crlu
pk
blt
ce c
S
t
manager jss
org
s
layout
generic
base
style xul
base
src
tr
tables
svg
base re
mathml
base
src
forms
prin in ht bu
js
src
xpconnect
src test
l
liveco
fdlib
tamarin
core
pcre code MM
shell pl e
jsd
mailnews
base
src util
search
src
imap
src
addrbook
src
mime
src
compose
src
import
outl
eud
src
oex
local
src
mapi
old ma
news
src
exten
palm
b s
db b
modules
oji
tests
src
JNI
C Arr
Ac
A C
sr
pu
plugin
tools
test s s
base
src
sam
def
S
libimg
png
rdf
src
libfont
jmcge
src
softupd
src
libpr0n
dec s
zlib
src
libre
src
libjar
libp
src
libb
src
xml
pro
s li
gfx
src
ps
gtk
xlib
windo
mac
os2
x11sh
theb xlib
qt phot
be
xpr
xp sh
f
cairo
cairo
src
glitz
src
libpixma
src
thebe
src
publi
publ
content
base
src p
html
content
src
doc
src
xslt
src
xslt xpath
xul
temp
src
doc
src
cont
svg
content
src
events
src
xbl
src
xml
d
xtf
can
extensions
canvas3d
src
xforms
webservice
soap
sche
pro
w i
java
xpcom
xmlterm
base line
python
xpco
spellch
src
walle
src
univ
src
sche
src
met
sr
sql
b
pre ins typ
aut
coo
xm
w s p
s
l
nsprpub
pr
src
md
wi uni ma
o b
misc
io
pthre
thr cp
li
tests
include
md p
li
t
li
xpcom
io glue
reflect
xptcal
src
x
s
ds
obsolete
c
string
pu sr
typelib
xpi x
base tests
build
MoreFi
compo
thr pr
s
widget
src
mac gtk2
windows
gtk
os2 beos
xpwi
xlib
qt ph
co g g
embedding
browser
activex
src
co
plu
co
pl
gtk
src
powerp
phot
web
qt
compon
printin
win
web
fi
qa
teste
tests
mfc w
directory
c-sdk
ldap
libraries
libldap
clie
exa
i
suncsdk
c-sdk
ldap
libraries cli i
xp
b
db
sqlite3
src
mork
src
tri
sr
md
ef
Compiler
Code
md
x
Front
Primi
Utilitie
Gener
zli qa
c
Runtim
Syste
C N C
gc
sr i
Tools
Pack
co D
Ex D
xpinstall
wizard
windows
setup uni
os2
setup
libxpne
GUSI
mac
unix
src
intl
uconv
ucvlat src
ucvcn
uti ucv
u
u
u t
unichar
src
locale
src
chardet
src
ctl
src
l s
netwerk
base
src
protocol
http
src
ftp
streamco
cache
test
dns
co
bui
s
java
webclient
src_moz
xpcom
te ja u
pluggab
wf
do
jni
plu
editor
libeditor
html base
text
txm
com
txtsv
toolkit
components
place
src
his s
do pa
airbag
airbag
xre m
xpfe
compone
sear boo
hi
st
bootstra
app
appshel
calendar
libical
src
libical
au
libic
test
base
js2
src
re
browser
components
places
src
migrat
boo s
parser
htmlparser
src p
expa
lib
tools
trace-
li
codes re
re
le
d
jp
f p
accessible
src
atk bas ht
ms
xu
dom
src
base
other-license
7zst
src
7zi
libart_
atk-1.
jpeg
msgsdk
C
protocol
cck
expat
driver
muc
ib
gc
boehm
c
plugin
oji
MRJ
plu
MRJ
pl
rdf
base
src
chro
d
docshell
base s
uriloader
extha b
camino
src
ipc
ipcd
e
lib
mac
profile
sr
config
mston
src
view
src
com
buil
win
caps
src
dbm
s i
sun
stu
web
w
sto
sr
gcon
chro
mini
Mozilla Vulnerabilities
security
nss
lib
libpkix
pkix_pl_nss
modu pki sy
pkix
top uti r
incl
ckfw
builtins ca
ns
freebl
mpi ecl
softoken
ssl
pk11wr
util certd smim
crmf
pki1
pki
pkcs12
certhig
pkcs de
jar cry
bas asn
cmd
zlib lib m pk si fips
crlu
pk
blt
ce c
S
t
manager jss
org
s
layout
generic
base
style xul
base
src
tr
tables
svg
base re
mathml
base
src
forms
prin in ht bu
js
src
xpconnect
src test
l
liveco
fdlib
tamarin
core
pcre code MM
shell pl e
jsd
mailnews
base
src util
search
src
imap
src
addrbook
src
mime
src
compose
src
import
outl
eud
src
oex
local
src
mapi
old ma
news
src
exten
palm
b s
db b
modules
oji
tests
src
JNI
C Arr
Ac
A C
sr
pu
plugin
tools
test s s
base
src
sam
def
S
libimg
png
rdf
src
libfont
jmcge
src
softupd
src
libpr0n
dec s
zlib
src
libre
src
libjar
libp
src
libb
src
xml
pro
s li
gfx
src
ps
gtk
xlib
windo
mac
os2
x11sh
theb xlib
qt phot
be
xpr
xp sh
f
cairo
cairo
src
glitz
src
libpixma
src
thebe
src
publi
publ
content
base
src p
html
content
src
doc
src
xslt
src
xslt xpath
xul
temp
src
doc
src
cont
svg
content
src
events
src
xbl
src
xml
d
xtf
can
extensions
canvas3d
src
xforms
webservice
soap
sche
pro
w i
java
xpcom
xmlterm
base line
python
xpco
spellch
src
walle
src
univ
src
sche
src
met
sr
sql
b
pre ins typ
aut
coo
xm
w s p
s
l
nsprpub
pr
src
md
wi uni ma
o b
misc
io
pthre
thr cp
li
tests
include
md p
li
t
li
xpcom
io glue
reflect
xptcal
src
x
s
ds
obsolete
c
string
pu sr
typelib
xpi x
base tests
build
MoreFi
compo
thr pr
s
widget
src
mac gtk2
windows
gtk
os2 beos
xpwi
xlib
qt ph
co g g
embedding
browser
activex
src
co
plu
co
pl
gtk
src
powerp
phot
web
qt
compon
printin
win
web
fi
qa
teste
tests
mfc w
directory
c-sdk
ldap
libraries
libldap
clie
exa
i
suncsdk
c-sdk
ldap
libraries cli i
xp
b
db
sqlite3
src
mork
src
tri
sr
md
ef
Compiler
Code
md
x
Front
Primi
Utilitie
Gener
zli qa
c
Runtim
Syste
C N C
gc
sr i
Tools
Pack
co D
Ex D
xpinstall
wizard
windows
setup uni
os2
setup
libxpne
GUSI
mac
unix
src
intl
uconv
ucvlat src
ucvcn
uti ucv
u
u
u t
unichar
src
locale
src
chardet
src
ctl
src
l s
netwerk
base
src
protocol
http
src
ftp
streamco
cache
test
dns
co
bui
s
java
webclient
src_moz
xpcom
te ja u
pluggab
wf
do
jni
plu
editor
libeditor
html base
text
txm
com
txtsv
toolkit
components
place
src
his s
do pa
airbag
airbag
xre m
xpfe
compone
sear boo
hi
st
bootstra
app
appshel
calendar
libical
src
libical
au
libic
test
base
js2
src
re
browser
components
places
src
migrat
boo s
parser
htmlparser
src p
expa
lib
tools
trace-
li
codes re
re
le
d
jp
f p
accessible
src
atk bas ht
ms
xu
dom
src
base
other-license
7zst
src
7zi
libart_
atk-1.
jpeg
msgsdk
C
protocol
cck
expat
driver
muc
ib
gc
boehm
c
plugin
oji
MRJ
plu
MRJ
pl
rdf
base
src
chro
d
docshell
base s
uriloader
extha b
camino
src
ipc
ipcd
e
lib
mac
profile
sr
config
mston
src
view
src
com
buil
win
caps
src
dbm
s i
sun
stu
web
w
sto
sr
gcon
chro
mini
Mozilla Vulnerabilities
js
src
xpconnect
src
livecon
content
base
src public
html
content
src
documen
src
xbl
src
xul
document
src p
content
src
templates
src
events
src
public
xslt
src
xslt base
xml
document
src
content
src
svg
conte
canvas
src
layout
xul
base
src
tree
src
grid
generic base
forms tables style
build
svg
mathm
printin
html
security
nss
lib
util
softoken
pki
ssl
cryptohi
free
pk1
cert
smi
cmd
manager
boot ssl
dom
src
base js
e
public
widget
src
mac windows
gtk
gtk2
xlib os2
qt phot beos
xpwi
pu
netwerk
protocol
http
src
about view
base
src
streamc
convert
cache
src
dn
s
caps
src inclu
xpinstall
src
uriloader
exthandler
un
os2
mac
wi be
base
modules
plugin
base
libpr0n
decod
gif x
libjar
oji
xpcom
string
public src
io glue
parser
expat
lib
htmlp
src
docshell
base
mailnews
base
src se
addrb
src
mime
src
ne
sr
embedding
components
windoww
src
c
br
w
editor
libeditor
html ba
te
c
xpfe
appshell
src
co
se
hi
extensions
xforms sq
b
toolkit
components
satch hi au
gfx
cairo
cairo
sr
intl
unich
sr ut
uc
sr
view
src p
acces
src
b x
rdf
base
chro
storage
src
chrome
src
db calen
brows
camin
ipc
webs
Mozilla Vulnerabilities
js
src
xpconnect
src
livecon
content
base
src public
html
content
src
documen
src
xbl
src
xul
document
src p
content
src
templates
src
events
src
public
xslt
src
xslt base
xml
document
src
content
src
svg
conte
canvas
src
layout
xul
base
src
tree
src
grid
generic base
forms tables style
build
svg
mathm
printin
html
security
nss
lib
util
softoken
pki
ssl
cryptohi
free
pk1
cert
smi
cmd
manager
boot ssl
dom
src
base js
e
public
widget
src
mac windows
gtk
gtk2
xlib os2
qt phot beos
xpwi
pu
netwerk
protocol
http
src
about view
base
src
streamc
convert
cache
src
dn
s
caps
src inclu
xpinstall
src
uriloader
exthandler
un
os2
mac
wi be
base
modules
plugin
base
libpr0n
decod
gif x
libjar
oji
xpcom
string
public src
io glue
parser
expat
lib
htmlp
src
docshell
base
mailnews
base
src se
addrb
src
mime
src
ne
sr
embedding
components
windoww
src
c
br
w
editor
libeditor
html ba
te
c
xpfe
appshell
src
co
se
hi
extensions
xforms sq
b
toolkit
components
satch hi au
gfx
cairo
cairo
sr
intl
unich
sr ut
uc
sr
view
src p
acces
src
b x
rdf
base
chro
storage
src
chrome
src
db calen
brows
camin
ipc
webs
Mozilla Vulnerabilities
js
src
xpconnect
src
livecon
content
base
src public
html
content
src
documen
src
xbl
src
xul
document
src p
content
src
templates
src
events
src
public
xslt
src
xslt base
xml
document
src
content
src
svg
conte
canvas
src
layout
xul
base
src
tree
src
grid
generic base
forms tables style
build
svg
mathm
printin
html
security
nss
lib
util
softoken
pki
ssl
cryptohi
free
pk1
cert
smi
cmd
manager
boot ssl
dom
src
base js
e
public
widget
src
mac windows
gtk
gtk2
xlib os2
qt phot beos
xpwi
pu
netwerk
protocol
http
src
about view
base
src
streamc
convert
cache
src
dn
s
caps
src inclu
xpinstall
src
uriloader
exthandler
un
os2
mac
wi be
base
modules
plugin
base
libpr0n
decod
gif x
libjar
oji
xpcom
string
public src
io glue
parser
expat
lib
htmlp
src
docshell
base
mailnews
base
src se
addrb
src
mime
src
ne
sr
embedding
components
windoww
src
c
br
w
editor
libeditor
html ba
te
c
xpfe
appshell
src
co
se
hi
extensions
xforms sq
b
toolkit
components
satch hi au
gfx
cairo
cairo
sr
intl
unich
sr ut
uc
sr
view
src p
acces
src
b x
rdf
base
chro
storage
src
chrome
src
db calen
brows
camin
ipc
webs
Distribution of MFSAs
Number of MFSAs
Num
ber o
f Com
pone
nts
1 3 5 7 9 11 13
12
520
5030
0
Distribution of Bug Reports
Number of Bug Reports
Num
ber o
f Com
pone
nts
1 3 5 7 9 13 17 24
12
520
5030
0
Imports
• 9,066 imports
• 79,541 import relations (x imports y)
• Takes about five minutes to compute
Resultssoon
Support VectorMachines
Support VectorMachines
Support Vectors
Support VectorMachines
ResultsNow!
Experiments
• 40 random splits6,968 rows in training set, 3,484 rows in validation set
• ClassificationTrain SVM, compute recall and precision
• RegressionTrain SVM, compute rank correlation on top 1%
• SVM: linear kernel with default parametersR implementation (up to 10GB of main memory)
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
0.55 0.60 0.65 0.70 0.75
0.35
0.40
0.45
0.50
0.55
(a) Precision and Recall
Recall
Prec
ision
0.2 0.3 0.4 0.5 0.6 0.7
0.0
0.2
0.4
0.6
0.8
1.0
(b) Rank Correlation
Rank Correlation
Cum
ulat
ive
Dist
ribut
ion
●●
●●●●
●●
●●
●●
●●●●
●●●●
●●●●●●●●●
●●●●●
●●●
●●
●2/3 of all vulnerable components detected
2/3 of all vulnerable components detected45% (about 1/2) of predictions correct
moderately strong correlation (mostly significant at p < 0.01)
Similar Results for Bugs
Packages + Import relationships(Schröter et al, ISESE 2006)
Precision: 66.7% Recall: 69.4%
Binaries + Dependencies(Zimmermann/Nagappan @ Microsoft Research, 2006)
Precision: 64.4% Recall: 75.3%
PredictedRank
Component ActualRank
1 nsDOMClassInfo 3
2 SGridRowLayout 95
3 xpcprivate 6
4 jsxml 2
5 nsGenericHTMLElement 8
6 jsgc 3
7 nsISEnvironment 12
8 jsfun 1
9 nsHTMLLabelElement 18
10 nsHttpTransaction 35
top related