today are by and large unaware of thesecurity problems they can (un-knowingly) introduce into code, ataxonomy of coding errors shouldprovide a real tangible benefit to thesoftware security community.

This approach represents a strik-ing alternative to taxonomies of at-tack patterns1 or simple-mindedcollections of specific vulnerabilities(such as MITRE’s CVE, www.cve.mitre.org). Attack-based approachesare based on knowing your enemyand assessing the possibility of similarattack—they represent the black hatside of the software security equa-tion. However, a taxonomy of soft-ware security errors is more positivein nature—it’s most useful to thewhite hat side of the software secu-rity world. In the end, both ap-proaches are valid and necessary.

Although the taxonomy pro-posed here is incomplete and imper-fect, it provides an important firststep. It focuses on collecting com-mon errors and explaining them in away that makes sense to program-mers. This new taxonomy is madeup of two distinct kinds of sets,which we’re stealing from biology: aphylum (a type of coding error, suchas illegal pointer value) and a king-

dom (a collection of phyla thatshares a common theme, such asinput validation and representa-tion). Both kingdoms and phyla nat-urally emerge from a soup of codingrules relevant to enterprise software,and it’s for this reason that this taxon-omy is likely to be incomplete andmight lack certain coding errors. Insome cases, it’s easier and more effec-tive to talk about a category of errorsthan to talk about any particular at-tack. Although categories are cer-tainly related to attacks, they aren’tthe same as attack patterns.

On simplicity: Sevenplus or minus twoWe’ve all seen lots of security tax-onomies over the years, and they allshare one unfortunate property—they’re too complex. People aregood at keeping track of seven things(plus or minus two),2 so we used thisfigure as a hard constraint and triedto keep the number of kingdomsdown to seven (plus one). In order ofimportance to software security,these kingdoms are

• input validation and representation,• API abuse,• security features,

• time and state,• errors,• code quality,• encapsulation, and• environment.

Let’s look at each of these a littlemore closely.

Input validation and representationMetacharacters, alternate encodings,and numeric representations causeinput validation and representation prob-lems. Of course, sometimes peoplejust forget to do any input validation atall. If you do choose to do input vali-dation, use a white list, not a black list.

Big problems result from puttingtoo much trust in input: buffer over-flows, cross-site scripting attacks,SQL injection, cache poisoning, andbasically all the low-hanging fruitthe script kiddies love so much.

API abuseAn API is a contract between a callerand a callee: the most common formsof API abuse occur when the callerfails to honor its end of the contract.If a program fails to call chdir()after calling chroot(), for exam-ple, it violates the contract that speci-fies how to securely change the activeroot directory. Another good exam-ple of library abuse is expecting thecallee to return trustworthy DNS in-formation to the caller. In this case,the caller abuses the callee API bymaking certain assumptions about itsbehavior. Really bad people also vio-late the caller–callee contract fromthe other side—for example, if you

KATRINA

TSIPENYUK AND

BRIAN CHESS

FortifySoftware

GARY

MCGRAW

Cigital

Taxonomies can help software developers and security

practitioners understand the common coding mis-

takes that affect security. The goal is to help develop-

ers avoid making these mistakes and more readily

identify security problems whenever possible. Because developers

Seven Pernicious Kingdoms: A Taxonomy of Software Security Errors

PUBLISHED BY THE IEEE COMPUTER SOCIETY ■ 1540-7993/05/$20.00 © 2005 IEEE ■ IEEE SECURITY & PRIVACY 81

Building Security InEditor: Gary McGraw, gem@cigital.com

Building Security In

subclass SecureRandom and returna not-so-random value, you’re notfollowing the rules. API abuse cate-gories are very common.

Security featuresSoftware security isn’t security soft-ware. All the magic crypto fairy dustin the world won’t make your codesecure, but it’s also true that you candrop the ball when it comes to essen-tial security features. Let’s say you de-cide to use the Secure Sockets Layer(SSL) to protect traffic across the net-work, but you really screw things up.Unfortunately, this happens all thetime. When we chunk together secu-rity features, we’re concerned withtopics like authentication, accesscontrol, confidentiality, cryptogra-phy, privilege management, and allthat other stuff on the CISSP exam.This stuff is hard to get right.

Time and stateDistributed computation is abouttime and state—that is, for morethan one component to communi-

cate, state must be shared (some-how), which takes time. Playingwith time and state is the biggest un-tapped natural attack resource on theplanet right now.

Most programmers anthropo-morphize (or, more accurately, onlysolipsistically ponder) their work.They think of themselves as the singleomniscient thread of control, plod-ding along and carrying out the entireprogram in the same way they wouldif forced to do the job manually. Howquaint. Modern computers switchbetween tasks very quickly, and inmulti-core, multi-CPU, or distrib-uted systems, two events can occur atexactly the same time. Defects relatedto unexpected interactions betweenthreads, processes, time, and informa-tion rush to fill the gap between theprogrammer’s model of how a pro-gram executes and what happens inreality. Several interactions happenvia shared state—semaphores, vari-ables, the file system, the universe,and, basically, anything that can storeinformation.

ErrorsWant to break software? Throwsome junk at a program and see whaterrors you cause. Errors are not only agreat source of “too much informa-tion” from a program, they’re also asource of inconsistent thinking thatcan be gamed. It gets worse: in mod-ern object-oriented systems, the no-tion of exceptions has reintroducedthe banned concept of goto. Alas.

Errors and error handlers repre-sent a class of programming contract,so in some sense, errors represent thetwo sides of a special form of API.However, the security defects re-lated to error handling are so com-mon they deserve a special kingdomall to themselves. As with API abuse,you can blow it in one of two ways:forgetting to handle errors at all (orhanding them so roughly that theyget all bruised and bloody) or pro-ducing errors that give out way toomuch information (to possible at-tackers) or are so radioactive that no-body wants to handle them.

Code qualitySecurity is a subset of reliability, justas all future TV shows are a subset ofmonkeys banging out scripts on zil-lions of keyboards. If you can com-pletely specify your system and all ofits positive and negative securitypossibilities, then security is a subsetof reliability. Poor code quality leads tounpredictable behavior, and from auser’s perspective, this often mani-fests itself as poor usability. For an at-tacker, bad quality provides anopportunity to stress the system inunexpected ways.

EncapsulationEncapsulation is about drawing strongboundaries around things and set-ting up barriers between them. In aWeb browser, this might mean en-suring that mobile code can’t whackyour hard drive arbitrarily. On a Webservices server, it might mean differ-entiating between valid authenti-cated data and mystery data foundstuck like yesterday’s gum to the bot-

82 IEEE SECURITY & PRIVACY ■ NOVEMBER/DECEMBER 2005

Building Security In

tom of a desk. Some of the most im-portant boundaries today come be-tween classes with various methods.Trust models require careful andmeticulous attention to bound-aries—in other words, keep yourhands off my stuff!

EnvironmentHey, it turns out that software runson machines with certain bindingsand connections to the bad, meanuniverse. Getting outside the soft-ware is important. This kingdom isthe kingdom of outside�in, and itincludes all the stuff outside yourcode that’s still critical to the securityof the software you create.

The phylaTo better understand the relation-ship between kingdoms and phyla,consider a recently discovered vul-nerability in Adobe Reader 5.0.x forUnix. The vulnerability is found in afunction called UnixAppOpen

FilePerform(), which copiesuser-supplied data into a fixed-sizestack buffer via a call tosprintf(). If the size of the user-supplied data is greater than the sizeof the buffer it’s copied into, impor-tant information (including the stackpointer) is overwritten. Thus bysupplying a malicious PDF file, anattacker can execute arbitrary com-mands on a target system. This attackis possible because of a simple codingerror—the absence of a check thatensures that the user-supplied data isno larger than the size of the destina-tion buffer. Developers associate thischeck with a failure to code defen-sively around the call tosprintf(). We classify it accord-ing to the attack it enables—bufferoverflow. In our taxonomy, wewould choose input validation and rep-resentation as the kingdom to whichthe buffer overflow phylum belongsbecause the lack of proper input val-idation is the attack’s root cause.(Due to space constraints, we can’tlist all the phyla here; visit http://vulncat.fortifysoftware.com for full

descriptions listed under their re-spective kingdoms.)

Static source-code analysis toolscan help detect phyla-representedcoding errors. Source-code analysisalso gives developers the opportu-nity to get quick feedback about thecode they’re writing as they’re writ-ing it. Our new taxonomy includescoding errors that occur in a varietyof programming languages, themost important of which are C andC++, Java, and the .NET family(including C# and ASP). Some ofthe phyla are language-specific be-cause the types of errors they repre-sent are applicable only to specificlanguages, and some are frame-work-specific (for the same reason).The phylum list as it exists is cer-tainly incomplete, but it’s adaptableto changes in the trends and discov-eries of new defects that are boundto happen over time, unlike variousexisting classification schemes.Classifying which errors are mostimportant to real-world enterprisedevelopers is the most importantgoal of this taxonomy—most of theinformation in it is derived from theliterature, various colleagues, andhundreds of customers.

Lists, piles, and collectionsThe idea of collecting and organiz-ing information about computer se-curity vulnerabilities has a longhistory, and several practitionershave even developed “top 10” listsand other related collections basedon experience in the field. The tax-onomy introduced here negotiates amiddle ground between rigorousacademic studies and ad hoc collec-tions based on experience.

Two of the most popular anduseful lists are the “19 sins” and the“Open Web Application SecurityProject (OWASP) top 10.” The firstlist is carefully described in the newbook 19 Deadly Sins of Software Secu-rity,3 and the OWASP Top 10 MostCritical Web Application SecurityVulnerabilities is available at www.

owasp.org/documentation/topten.html. The main limitation of bothlists is that they mix specific types oferrors and vulnerability classes andtalk about them all at the same levelof abstraction. The 19 sins, for ex-ample, include both “bufferoverflows” and “failing to protectnetwork traffic” categories at thesame level, even though the first is avery specific coding error, whereasthe second is a class comprising var-ious kinds of errors. Similarly,OWASP’s top 10 includes “cross-site scripting (XSS) flaws” and “in-secure configuration management”at the same level. This unnecessarylevel confusion is a serious problemthat can lead to confusion amongpractitioners. Although our classifi-cation scheme is quite differentfrom the lists described earlier, wecan easily map the categories thatcomprise these lists to our kingdoms(see Table 1).

T he seven pernicious kingdomsoffer a simple, effective organiz-

ing tool for software security codingerrors, and with more than 60clearly defined phyla,4 this taxon-omy is both powerful and useful. Buttaxonomy work is an ongoing quest,and further refinement and evolu-tion is necessary. Please send anyfeedback regarding this taxonomy tobrian@fortifysoftware.com.

References1. G. Hoglund and G. McGraw,

Exploiting Software: How to BreakCode, Addison-Wesley, 2004.

2. G. Miller, “The Magic NumberSeven, Plus or Minus Two,” ThePsychological Rev., vol. 63, 1956, pp.81–97; www.well.com/user/smalin/miller.html.

3. M. Howard, D. LeBlanc, and J.Viega, 19 Deadly Sins of SoftwareSecurity, McGraw-Hill, 2005.

4. K. Tsipenyuk, B. Chess, and G.McGraw, “Seven PerniciousKingdoms: A Taxonomy of Soft-ware Security Errors,” to be pub-

www.computer.org/security/ ■ IEEE SECURITY & PRIVACY 83

Building Security In

lished in Proc. NIST Workshop onSoftware Security Assurance Tools,Techniques, and Metrics (SSATTM),US Nat’l Inst. Standards and Tech-nology, 2005.

Katrina Tsipenyuk is the founding mem-ber of the Security Research Group at For-tify Software. She has a BS and an MS incomputer science with an emphasis onsecurity from the University of California,San Diego. Her thesis work includesresearch on mobile agent security. Con-tact her at katrina@fortifysoftware.com.

Brian Chess is chief scientist at FortifySoftware. His work focuses on practicalmethods for creating secure systems.Chess has a PhD in computer engineer-ing from the University of California,Santa Cruz, where he applied his back-ground in integrated circuit test and ver-ification to the problem of identifyingsecurity-relevant defects in software. Con-tact him at brian@fortifysoftware.com.

Gary McGraw is chief technology officerof Cigital. His real-world experience isgrounded in years of consulting withmajor corporations and software pro-ducers. McGraw is the coauthor ofExploiting Software (Addison-Wesley,2004), Building Secure Software (Addi-son-Wesley, 2001), Java Security (JohnWiley & Sons, 1996), and four otherbooks. He has a BA in philosophy fromthe University of Virginia and a dual PhDin computer science and cognitive sciencefrom Indiana University. Contact him atgem@cigital.com.

84 IEEE SECURITY & PRIVACY ■ NOVEMBER/DECEMBER 2005

KINGDOMS 19 SINS OPEN WEB APPLICATION SECURITY PROJECT (OWASP) TOP 10

Input validation and representation Buffer overflows, command injection, cross-site Buffer overflows, cross-site scripting flaws,

scripting, format string problems, integer range injection flaws, unvalidated input

errors, SQL injection

API abuse Trusting network address information

Security features Failing to protect network traffic, failing to store Broken access control, insecure storage

and protect data, failing to use cryptographically

strong random numbers, improper file access,

improper use of SQL, use of weak password-based

systems, unauthenticated key exchange

Time and state Signal race conditions, use of “magic” URLs and Broken authentication and session

hidden forms management

Errors Failure to handle errors Improper error handling

Code quality Poor usability Denial of service

Encapsulation Information leakage

Environment Insecure configuration management

Table 1. Mapping 19 sins and the OWASP top 10 to the proposed kingdoms.

Ensure that your networks operate safely and provide critical services even in the face

of attacks. Develop lasting security solutions, with this peer-reviewed publication.

Top security professionals in the field shareinformation you can rely on:

Wireless Security • Securing the Enterprise • Designing for Security Infrastructure Security • Privacy Issues

• Legal Issues • Cybercrime • Digital Rights Management • In-tellectual Property Protection and Piracy • The Security Profes-

sion • Education

Order your subscription today.

www.computer.org/security/

BE SECURE.

DON’T RUN THE RISK.

BE SECURE.

DON’T RUN THE RISK.