36

Internet routers look up the desti-nation address of an incoming packet in itsforwarding table to determine the packet’snext hop on its way to the final destination.This routing lookup operation takes place oneach arriving packet by every router in thepath that the packet takes from its source tothe destination.

The adoption of classless interdomain rout-ing (CIDR)1 in 1993 required a routinglookup to perform a longest prefix match oper-ation. A router maintains a set of destinationaddress prefixes in a forwarding table. Given apacket, the operation finds the longest prefixin the forwarding table that matches the firstfew bits of the packet’s destination address.

To provide enhanced services such as pack-et filtering, traffic shaping, policy-based rout-ing, and so on, routers also must be able torecognize flows. A flow is a set of packets thatobey some rule, also called a policy, on thepacket’s header fields. These fields includesource and destination Internet protocoladdresses, source and destination port num-bers, protocol, and others. For instance, allpackets with a specified destination IP addressand specified source IP address may be definedby a rule to form a single flow.

A collection of rules is called a policy data-base or a classifier. Identification of the flow ofan incoming packet is called packet classifica-tion and is a generalization of the routinglookup operation. Packet classification

requires the router to find the best-matchingrule among the set of rules in a given classifi-er that match an incoming packet. A rule mayspecify a prefix, range, or a simple regularexpression for each of several packet headerfields. An arriving packet’s header may satisfythe conditions of more than one rule—inwhich case the rule with the highest prioritydetermines the flow of the arriving packet.

At the time of this writing, improvementsin optical communication technologies suchas dense wavelength-division multiplexing(DWDM) have resulted in continuallyincreasing link speeds up to 40 Gbps perinstalled fiber. However, routers have beenlargely unable to keep up at the same pace; amaximum of 10 Gbps (OC192) ports areavailable now. One main reason for this is therelatively complex packet processing requiredat each router.

As a result, the problems of routing lookupand packet classification have recently receivedconsiderable attention, both in academia andthe industry. See, for example, the literature forsolutions to the routing lookup problem2-7 andfor solutions to the packet classification prob-lem.8-13 Many of these reports have indicatedthe difficulty of the general multidimensionalpacket classification problem in the worst case.

Hardware realizations of algorithmicallysimpler solutions such as linear or fully asso-ciative searches have found favor in some com-mercial deployments. A popular device is a

Devavrat ShahPankaj Gupta

Stanford University

ONE POPULAR HARDWARE DEVICE FOR PERFORMING FAST ROUTING LOOKUPS

AND PACKET CLASSIFICATION IS A TERNARY CONTENT-ADDRESSABLE MEMORY

(TCAM). WE PROPOSE TWO ALGORITHMS TO MANAGE THE TCAM SUCH THAT

INCREMENTAL UPDATE TIMES REMAIN SMALL IN THE WORST CASE.

0272-1732/01/$10.00 2001 IEEE

FAST UPDATING ALGORITHMSFOR TCAMS

special type of fully associative memory: aternary content-addressable memory(TCAM). Each cell in a TCAM can take threelogic states: 0, 1, or don’t-care X. A CAMallows a fully parallel search of the forwardingtable or a classifier database. The ternary capa-bility lets the TCAM store wild cards and vari-able-length prefixes by storing don’t-cares.Lookups are performed in a TCAM by stor-ing forwarding table entries in order of decreas-ing prefix lengths and choosing the first entryamong all the entries that match the incom-ing packet’s destination address. Packet classi-fication is carried out similarly by storingclassifier rules in order of decreasing priority.

The need to maintain a sorted list makesincremental updates slow in a TCAM. If N isthe total number of prefixes to be stored in aTCAM having M entries, naive addition of anew entry can result in the need to move O(N)TCAM entries to create the space required toadd the entry at a particular place in theTCAM to maintain the sorted order. Alter-natively, some TCAM entries can be inten-tionally left unused in anticipation of futureadditions. However, this leads to wasted spaceand underutilization of the TCAM. Besides,the worst case still remains O(N).

We were motivated by the desire to simul-taneously achieve fast incremental updates aswell as full use of the TCAM. With thisobjective, we describe worst-case algorithms(one specific for route lookups, and the othersuitable for both lookups and classification)that achieve the optimal number of aTCAM’s operations (such as move/write/read) required for an incremental update. Thealgorithms are online in the sense that theyperform operations on memory as updaterequests arrive, instead of batching severalupdate requests.

In particular, we show that, if L is the widthof the destination address field (L equals 32 inIPv4, and 128 in IPv6), no more than L/2memory operations are required. This algo-rithm is proved to be optimal; that is, it per-forms no worse than any other algorithm inthe worst case that keeps the list of forwardingtable entries in order of decreasing prefixlengths. This compares favorably with the Lmemory operations in the memory manage-ment schemes recommended by some TCAMvendors (see Sibercore Technologies,

www.sibercore.com/scan01_cidr_p03.pdf cur-rent Feb. 2000). See also our later discussion.

It turns out that it isn’t necessary to keep allthe forwarding table entries in the order ofdecreasing prefix lengths; instead, only over-lapping prefixes need to be in this order. Twoprefixes overlap if one is a prefix of the other;for example, 01* overlaps with 0101*, but notwith 001*. This observation is used in the sec-ond algorithm. Although not proved, this algo-rithm seems to be optimal in the number ofworst-case memory operations required tohandle a forwarding table update. We alsodescribe how this algorithm and the results forrouting lookups extend to packet classification.

To the best of our knowledge, there’s noprevious work that attempts to (algorithmi-cally) optimize updates on a TCAM. MostTCAM vendors live with an O(N) worst-caseupdate time solution. Some attempt to pro-vide a hardware maximum function that com-putes the maximum of the prefix lengths (orpriorities) of all matching entries, hence elim-inating the requirement of keeping the tableentries sorted. However, computing the max-imum of O(M)/log2 M-bit numbers is expen-sive in current technology in terms of logicarea and speed. (M is around 16K to 64Kpresently). This is likely to worsen in thefuture as TCAMs scale to greater densities.One recent paper14 uses circuit-level opti-mizations for fast updates at the cost of slow-er search time and lower memory density.

Note that while the algorithms we discusshave been mentioned in the context of a par-allel-search TCAM, they are equally applica-ble to other algorithms that keep a sorted listof forwarding table entries or classifier rules,such as hardware realizations of a linear searchalgorithm.

Longest prefix matching using TCAMsIP addresses are written in the dashed quad

notation, for instance, 103.23.3.1, represent-ing the four bytes of an IPv4 destinationaddress separated by dots. An entry in arouter’s forwarding table is a pair: <route-pre-fix,nextHop>. A route-prefix, or simply a pre-fix, is represented like an IP address but mayhave some trailing bits treated as wild cards;this denotes the aggregation of several 32-bitdestination IP addresses. For example, theaggregation of 256 addresses 103.23.3.0

37JANUARY–FEBRUARY 2001

through 103.23.3.255 is represented by theprefix 103.23.3/24, where 24 is the length ofthe prefix, and the last 8 bits are wild cards.Other examples of prefixes are 101/8,54.128/10, 38.23.32/21, and 200.3.41.1/32.nextHop is the IP address of a router or endhost that is a neighbor of this router.

Given an incoming packet’s destinationaddress, a routing lookup operation finds theentry with the longest—that is, the most spe-cific—of all the prefixes matching the first fewbits of the incoming packet’s destination

address. It then forwards theincoming packet to thisentry’s next-hop address.

This longest prefix match-ing operation is performed ina TCAM by storing entries indecreasing order of prefixlengths. The TCAM searchesthe destination address of anincoming packet with all theprefixes in parallel. Severalprefixes (up to L = 32 in thecase of IPv4 lookups) maymatch the destination address.A priority encoder logic then

selects the first matching entry—the entry withthe matching prefix at the lowest physicalmemory address. Figure 1 shows an example.

Figure 2 shows the general configurationfor storing N prefixes in a TCAM with Mmemory locations. We refer to the set of allprefixes of length j as Pj. We also assume amemory manager software that arranges pre-fixes in the desired order and sends appropri-ate instructions to the TCAM hardware.

Forwarding tables in routers are dynamic;prefixes can be added or deleted as links go upor down due to changes in network topology.These changes can occur at the rate of approx-imately 100 to 1,000 prefixes per second.15

While this is slow in comparison to the pack-et lookup rate (which is on the order of mil-lions of packets per second), it’s desirable toobtain quick TCAM updates. Slow updatesmay cause incoming packets to be bufferedwhile an update operation is being carried out.This is undesirable for many applicationsbecause it may cause head-of-line blocking andrequire a large buffer space separate from themain packet buffer memory in the router.Indeed, some TCAM vendors (see NetlogicMicrosystems at www.netlogicmicro.com) usea single-cycle update time for a big competitiveadvantage. Hence, it’s desirable to keep theincremental update time as small as possible.

Forwarding table updates complicate keep-ing the list of prefixes in the TCAM in sortedorder. This issue is best explained with the exam-ple in Figure 1. Assume that a new prefix103.23.128/18 is to be added to the forwardingtable. It must be stored between prefixes103.23.3/24 (P1) and 103.23/16 (P2), cur-rently at memory locations 0 and 1 to maintain

38

TERNARY CAMS

IEEE MICRO

Memorylocation

Priorityencoder

Prefix Next hop

0

1

2

3

4

5

6

7

P1

P2

P3

P4

P5

103.23.3/24

103.23/16

101.1/16

101.20/13

100/9

171.3.2.22

171.3.2.4

120.33.32.98

320.3.3.1

10.0.0.111

103.23.3.7 P1171.3.2.22

Figure 1. Longest prefix matching using a TCAM.

31-bit prefixes

30-bit prefixes

9-bit prefixes

32-bit prefixes

8-bit prefixesM − 1

0

Figure 2. General configuration of a TCAMused for longest prefix matching. No prefixesof a length less than 8 bits are shownbecause they are typically not found in for-warding tables.

the sorted order. However, there’s a problemsince there’s no empty space at that location.There can be several ways to handle this issue.

The TCAM manager can keep the free spacepool (containing all unused TCAM entries) atone end of the TCAM, say at the bottom, asshown in Figure 3. A naive solution would shiftprefixes P2 to P5 downward in memory by onelocation each, thus creating an empty spacebetween P1 and P2 to store the new prefix. Thishas worst-case time complexity O(N), whereN is the number of prefixes in the TCAM ofsize M, and is clearly expensive. For instance, ifN = 64,000, it will take 1.2 milliseconds(assuming one memory write operation can beperformed in a 20-ns clock cycle) to completeone update operation. This is too slow for alookup engine that completes one lookup in20 ns because a large packet buffer would berequired to store incoming packets while anupdate is being completed.

In anticipation of additions and deletionsof prefixes, the TCAM may keep a few emptymemory locations at all X nonempty memo-

ry locations, as shown in Figure 4. The aver-age case update time improves to O(X) butdegenerates to O(N) if the intermediate emptyspaces are filled up. This solution also wastesprecious CAM space.

The following solution is based on theobservation that two prefixes of the samelength don’t need to be in any specific order.This means that if j is larger than k, all pre-fixes in the set Pj must appear before those inthe set Pk, but prefixes within set Pj mayappear in any order. Hence, there’s only a par-tial ordering constraint between all prefixes(as opposed to a complete ordering constraintin the naive solution). We call this constraintthe prefix-length ordering constraint.

This observation leads to an algorithm,referred to here as the L-algorithm, that cancreate an empty space in a TCAM in no morethan L memory shifts (recall that L = 32 ), asshown in Figure 5 (next page). The averagecase can be improved again by keeping someempty spaces that are not all at the bottom ofthe TCAM. A later section proposes an opti-mal algorithm, PLO_OPT, that brings downthe worst-case number of memory operations

39JANUARY–FEBRUARY 2001

31-bit prefixes

30-bit prefixes

9-bit prefixes

32-bit prefixes

8-bit prefixes

0

Empty space

M − 1

N − 1

Figure 3. This naive solution keeps the freespace pool at the bottom of memory.

Empty space

31-bit prefixes

30-bit prefixes

9-bit prefixes

32-bit prefixes

8-bit prefixes

0

M − 1

Figure 4. This solution improves the averagecase update time by keeping empty spacesinterspersed with prefixes in the TCAM.

per update to L/2.The prefix-length ordering constraint is also

more restrictive than that required for a cor-rect longest prefix matching operation usinga TCAM. In Figure 1, while prefix103.23.3/24 (P1) needs to be at a lower mem-ory address than prefix 103.23/16 (P2) at alltimes, it can be stored anywhere in the TCAMwith respect to prefixes P3, P4, and P5. Thisis because P1 doesn’t overlap with prefix P3or P4 or P5. That is, no incoming destinationaddress can match both P1 and P3, or P1 andP4, or P1 and P5. Hence, the constraint onthe ordering of the prefixes in a TCAM cannow be relaxed to only overlapping prefixes.Since two prefixes overlap if one is fully con-tained inside the other, there’s an orderingconstraint between two prefixes pi and pj if andonly if one is a prefix of the other.

If all prefixes were to be visualized as beingstored in a trie data structure, only prefixes thatlie on the same chain (the path from the rootto a leaf node) of the trie need to be ordered.

For example, as shown in Figure 6, prefixesQ3, Q2, and Q1 must appear in order sincethey lie on the same chain. Prefix Q4 can bestored anywhere with respect to Q2 and Q3,but it must be stored at a lower memory loca-tion than Q1. We refer to this constraint as thechain-ancestor ordering constraint. A later sec-tion proposes an algorithm, CAO_OPT, thatexploits this relaxed constraint to decrease theworst-case number of memory operations perupdate to D/2. Here, D is the maximumlength of any chain in the trie. D is usuallysmall (at most 5) for even large backbone for-warding tables, hence, this algorithm achievesworst-case updates in a few clock cycles.

Prefix-length ordering constraint algorithmThe basic idea of the PLO_OPT algorithm

is to keep all the unused entries in the centerof the TCAM. The arrangement (shown inFigure 7) is such that the set of prefixes oflength L, L − 1, ..., L/2 are always above thefree space pool, and the set of L/2 − 1, L/2 −2, ..., 1 prefixes are always below the free spacepool. Addition of a new prefix would have toswap at most L/2 memory entries to obtain anunused memory entry. Deletion of a prefix isexactly the reverse of addition, moving thenewly created space back to the center of theTCAM. To support the update operations, thealgorithm uses a trie data structure to keeptrack of the prefixes stored in the TCAM.

The average case update time can again beimproved to better than L/2 by keeping someunused entries near each set Pi , as was done inFigure 4. The worst-case number of memoryoperations is now at least L/2 and can become

40

TERNARY CAMS

IEEE MICRO

31-bit prefixes

30-bit prefixes

9-bit prefixes

32-bit prefixes

8-bit prefixes

0

Empty space

M − 1

N − 1

Figure 5. The prefix-length ordering con-straint enables an empty memory location tobe found in most L = 32 memory shifts.

Q1

Q4Q3

Q2

Figure 6. Illustration of the chain-ancestorordering constraint. There are two maximalchains in this trie: one comprises Q1, Q2, andQ3; the other comprises Q1 and Q4.

even higher. The distribution of the number ofunused entries to be kept around Pi dependson the distribution of updates and is thereforedifficult to determine a priori. Possible heuris-tics for placement of an empty space include auniform distribution and a distribution learnedfrom recently observed update requests.

The PLO_OPT algorithm can be provedto be an optimal online algorithm under theprefix-length ordering constraint. In otherwords, no algorithm that is unaware of futureupdate requests can perform better than thePLO_OPT algorithm under the prefix-lengthordering constraint.

Algorithm for chain-ancestor ordering constraintBefore we describe the CAO_OPT algo-

rithm, we need to clarify some terminology.

• LC(p) is the longest chain comprising

prefix p.• len(LC(p)) is length of (number of pre-

fixes in LC(p)).• rootpath(p) is the path from the trie root

node to node p.• ancestor of p is any node in rootpath(p).• prefix-child of p is a child node of p that

has a prefix.• hcld(p) is highest prefix-child of p; that

is, among the children of p, the node thathas the highest memory location in theTCAM.

• HCN(p) is the chain comprising ances-tors of p, prefix p itself, hcld(p),hcld(hcld(p)), and so on; that is, a descen-dant node of p is in HCN(p) if it’s thehighest prefix-child of its ancestor.

The CAO_OPT algorithm also keeps thefree space pool in the center of the TCAMwhile maintaining the chain-ancestor orderingamong the entries in the TCAM. Hence, a log-ical inverted trie can be superimposed on theprefixes stored in the TCAM. For example, theprefixes in Figure 6 may be stored as shown inFigure 8, with the logical inverted trie shown indashed lines. The basic idea is to arrange the

41JANUARY–FEBRUARY 2001

31-bit prefixes

32-bit prefixes

0

Empty space

17-bit prefixes

16-bit prefixes

1-bit prefixes

2-bit prefixes

3-bit prefixes

M − 1

N − 1

Figure 7. The PLO_OPT algorithm keeps allthe unused TCAM entries in the center of theTCAM such that all prefixes longer than 16bits are above the empty space, and all pre-fixes shorter than 16 bits are below theempty space at all times.

Q4

Q2

Q3

Q1

0

M − 1

Figure 8. Memory assignment of prefixes inFigure 6 under the chain-ancestor orderingconstraint. Also shown is the logical invertedtrie.

chains in such a way so as to maintain the fol-lowing invariant. Assume that D = len(LC(p))for a prefix p. Every prefix p is stored in a mem-

ory location such that there are at most D/2prefixes between p and a free space entry in theTCAM. Basically, the algorithm distributes themaximal trie chains around the free space poolas equally as possible (see Figure 9).

InsertionInsertion of a new prefix q proceeds in the

following manner. First, LC(q) is identifiedusing an auxiliary data structure that’sdescribed later. It must be determined whetherq needs to be inserted above or below the freespace pool (to maintain the balance of LC(q)).The two cases are handled separately.

Case I (Figure 10). Assume that q is to beinserted above the free space pool betweenprefixes pi and pi + 1 on LC(q). One emptyunused entry can be created at that location bymoving prefixes on LC(q) downward one byone, starting from pj to the unused entry ateither the top (memory location labeled m1in Figure 10 or the bottom (m2) of the freespace pool. The total number of movementsis clearly less than D/2, where D is len(LC(q)).The movements don’t violate the chain-ances-tor ordering constraint since a prefix is moveddownward after its ancestor has moved.Hence, the constraint is always satisfied.

Case II (Figure 11). Assume that q is to be

42

TERNARY CAMS

IEEE MICRO

p

Free space pool

hcld (hcld(p))

hcld(p)

pj

pi

pi + 1

LC(p)

0

M − 1

Figure 9. The distribution of chains in theTCAM under the chain-ancestor orderingconstraint. Every prefix p is at a distance lessthan or equal to D/2 prefixes for the freespace pool, where D equals len(LC(p)).

Free space pool

Insertq here

pj

pi

pi + 1

LC(q)

0

M − 1

M1

M2

Figure 10. The way an insertion proceeds inthe CAO_OPT algorithm when the prefix tobe inserted is above the free space pool.

Insertq here

LC(q)

HCN(q)

0

M − 1

M1

M2Free space pool

Figure 11. The way an insertion proceeds inthe CAO_OPT algorithm when the prefix tobe inserted is below the free space pool.

inserted below the free space pool. Creating anempty entry in the TCAM now requires mov-ing the prefixes upward toward the free spacepool. Hence, the chain we consider in this caseis HCN(q), which may or may not be identi-cal to LC(q). Movement of prefixes one by oneupward doesn’t violate the chain-ancestorordering constraint since a prefix is moved tothe location previously occupied by the childthat occupied the highest memory locationamong all the children. Again, the total num-ber of movements is clearly less than D/2.

DeletionDeletion is similar to insertion, with the fol-

lowing exceptions:

1. It works in reverse, moving the newly cre-ated empty space to the free space pool.

2. It works on the chain that has prefix padjacent to the free space pool; that is,prefix p is at memory locations m1 − 1 orm2 + 1.

Figure 12 shows the deletion of a prefix.The new unused entry created by the deletionof prefix q is rippled up by moving prefixesdownward on this chain. The total numberof movements is less than D/2, where D isnow either the length of LC(p) if q is deleted

from below the free space pool, or the lengthof HCN(p) if q is deleted from above the freespace pool.

Auxiliary trie data structure The CAO_OPT algorithm maintains an

auxiliary trie data structure similar toPLO_OPT to support update operations.However, to determine LC(p) and HCN(p)quickly, we need to keep more information intrie node p. This takes no more than O(L)time by maintaining the following addition-al fields in every trie node: wt(p), wt_ptr(p),and hcld_ptr(p). wt(p) equals

Here, lchild(p) and rchild(p) are the immedi-ate left and right children nodes of p. wt_ptr(p)keeps a pointer to the prefix child, which hasthe highest weight, and hcld_ptr(p) keeps apointer to the prefix child, which appears atthe highest memory location in the TCAM.

Although we haven’t proved it, we conjec-ture that the CAO_OPT algorithm is an opti-mal online algorithm under the chain-ancestorordering constraint.

Simulation resultsIn our simulation we used two publicly

available routing-table snapshots (at MAE-EAST and MAE-WEST network accesspoints) and three-hour BGP-update traces onthese snapshots taken from Merit(www.merit.edu/ipma/routing_table). Table1 lists the statistics of the routing tables andBGP updates.

1

1

if p is a leaf

wt lchild p rchild p

if p isnot a leaf and not a prefix

wt lchild p rchild p

otherwise

max ,

max ,

( ) ( )( )( )+ ( ) ( )( )( )

43JANUARY–FEBRUARY 2001

0

M − 1

M1

M2Free space pool

q

Delete q

p

Figure 12. Deletion of a prefix in theCAO_OPT algorithm.

Table 1. Statistics of routing tables

and update traces used

in simulations.

Type MAE-EAST MAE-WEST

Prefixes 43344 35217Inserts 34204 34114Deletes 9140 1103

Figure 13 shows a running average of thenumber of memory movements (memorywrites or shifts) required in the L-algorithm as

a function of the number ofupdates. The figure shows thatthe average settles down toaround eight memory move-ments per update operation.This is expected since most ofthe updates happen to prefix-es that are between 8 bits and24 bits long, because there arevery few (less than 0.1%) pre-fixes that are longer than 24bits. Hence, if we assume thatupdates are uniformly distrib-uted between these lengths,the running average shouldsettle at (24 − 8)/2 = 8. Asshown in Figure 14, the aver-age drops to approximatelyfour memory movements forthe PLO_OPT algorithm.This is again expected sincetheoretical analysis showed animprovement over the L-algo-rithm by a factor of 2.

The motivation for a lessstringent constraint (thechain-ancestor ordering con-straint) is clear from Figure15, which plots the maximalchain length distribution ofthe two routing tables. Thefigure shows exponentiallydecreasing distributions; forexample, 97% of the MAE-EAST chains have a lengthless than or equal to two, andall chains have a length lessthan six.

Figure 16 plots the runningaverage of the number ofmemory movements requiredas a function of the numberof updates using theCAO_OPT algorithm. Thisfigure shows that the averagequickly drops down to 1.02to 1.06 for both routingtables.

Table 2 and Table 3 (onpage 46) list performance

summary statistics of both algorithms on thetwo routing tables. Note that the standarddeviation of the CAO_OPT algorithm is

44

TERNARY CAMS

IEEE MICRO

0

1

2

3

4

5

6

7

8

9

10

11

50 100 150 200 250 300 350 400 450

Number of updates (x100)

MAE-WESTMAE-EAST

Num

ber

of m

emor

y m

ovem

ents

Figure 13. The running average of the number of memory movements required by the L-algorithm to support updates on MAE-WEST and MAE-EAST routing tables.

Num

ber

of m

emor

y m

ovem

ents

500 100 150 200 250 300 350 400 450

Number of updates (x100)

MAE-WESTMAE-EAST

4.6

4.4

4.2

4.0

3.8

3.6

3.4

3.2

3.0

2.8

2.6

Figure 14. The running average of the number of memory movements required by thePLO_OPT algorithm to support updates on MAE-WEST and MAE-EAST routing tables.

quite small (and much less than that of thePLO_OPT algorithm). This is probably due

to the exponentially decreasing chain lengthdistribution. This should make the

45JANUARY–FEBRUARY 2001

1

10

100

1,000

10,000

100,000

1.5 2 2.5 3 3.5 4 4.5 5

Maximal chain length

Num

ber

of c

hain

s

MAE-WESTMAE-EAST

Figure 15. The chain length distribution on the two routing tables. Note the logarithmic scaleon the y-axis.

1

1.02

1.04

1.06

1.08

1.1

1.12

1.14

1.16

1.18

0 50 100 150 200 250 300 350 400 450

Number of updates (x100)

Num

ber

of m

emor

y m

ovem

ents

MAE-WESTMAE-EAST

Figure 16. The running average of the number of memory movements required by theCAO_OPT algorithm to support updates on MAE-WEST and MAE-EAST routing tables.

CAO_OPT algorithm even more attractivein practice.

Packet classificationSo far, we’ve discussed updates in the con-

text of routing lookups. Both the PLO_OPTand CAO_OPT algorithms also extend topacket classification.

The prefix-length ordering constraint isequivalent to keeping the list of rules in a clas-sifier ordered by priority. The PLO_OPTalgorithm then degenerates to the naive algo-rithm that requires O(N) memory movementsper update in the worst case. Analyzing for theset of overlapping rules and generating a con-straint tree could bring this down. Two rulesoverlap if a packet exists that matches bothrules, and only overlapping rules need to bekept in the order of their priority in theTCAM. The constraint tree captures theseconstraints in a tree form.

Using the constraint tree to determine ruleordering instead of the prefix trie lets us usethe CAO_OPT algorithm with little modifi-cation. Of course, the benefit of using thechain-ancestor ordering constraint dependson the chain length distribution in the con-straint tree, and can only be determined bydoing an analysis of real-life classifiers. Thistask is made difficult by the absence of largepublicly available classifiers.

Handling incremental updates in routinglookups can be a slow process—even in

simple data structures such as that maintained

in a ternary CAM. Both of our proposed algo-rithms for high-speed updates in TCAMsoperate under two separate constraints. Nei-ther requires additional circuitry on theTCAM chip, and one can be proved optimal.In particular, the proposed PLO_OPT algo-rithm for the stricter (and more well-known)prefix-length ordering constraint improvesupdate speed by a factor of two over the best-known solution.

The CAO_OPT algorithm for the less strin-gent chain-ancestor ordering constraint guar-antees correctness at all times, and completesone prefix update in slightly greater than one(1.02 to 1.06 observed using simulations onreal-life routing tables and update traces) mem-ory movement per update operation. Algo-rithm CAO_OPT is also useful for fast updateswhen a TCAM is used for packet classification.

The algorithms described here assume thatthe maximum number of entries is almost thesame as the TCAM size. However, we are nowinterested in whether we can provide betterbounds if we’re guaranteed that the TCAMwon’t be occupied more than (1/k)th fractionof its size. If we can provide better bounds,can we prove optimality? We could simplyextend our two algorithms to a case in whichthe bounds are reduced from d/2 to d/2k.Then, we can provide optimality under cer-tain assumptions, but not for the general case.Moreover, it’ll be interesting to see if we canachieve an optimal average case algorithmunder certain updating distribution. MICRO

AcknowledgmentsWe gratefully acknowledge Spencer Greene,

now at Juniper Networks, for mentioning thepossibility of a better algorithm with a lessstringent constraint than the prefix-lengthordering constraint.

References1. Y. Rekhter and T. Li, “An Architecture for IP

Address Allocation with CIDR,” RFC 1518,1993; http://rfc.net/rfc1518.html.

2. M. Waldvogel et al., “Scalable High-SpeedIP Routing Lookups,” Proc. ACM Sigcomm,ACM, N.Y., 1997, pp. 25–36.

3. A. Brodnik et al., “Small Forwarding Tablesfor Fast Routing Lookups,” Proc. ACM Sig-comm, ACM, N.Y., 1997, pp. 3–13.

4. P. Gupta, S. Lin, and N. McKeown, “Rout-

46

TERNARY CAMS

IEEE MICRO

Table 2. Summary of performance numbers on

MAE-WEST routing table.

Algorithm Maximum Average Standard deviation

L-algorithm 22.0 7.76 3.93PLO_OPT 13.0 3.56 1.93CAO_OPT 3.0 1.05 0.02

Table 3. Summary of performance numbers on

MAE-EAST routing table.

Algorithm Maximum Average Standard deviation

L-algorithm 21.0 7.27 4.09PLO_OPT 12.0 4.1 2.03CAO_OPT 3.0 1.02 0.01

ing Lookups in Hardware at Memory AccessSpeeds,” Proc. INFOCOM, IEEE Press, Pis-cataway, N.J., 1998, pp. 1240–1247.

5. B. Lampson, V. Srinivasan, and G. Varghese,“IP Lookups Using Multiway and Multicol-umn Search,” Proc. INFOCOM, 1998, pp.1248–1256.

6. S. Nilsson and G. Karlsson, “IP-AddressLookup Using LC-Tries,” IEEE J. SelectedAreas in Communications, vol. 17, no. 6,1999, pp. 1083–1092.

7. V. Srinivasan and G. Varghese, “FastAddress Lookups Using Controlled PrefixExpansion,” ACM Trans. Computer Sys-tems, vol. 17, no. 1, Oct. 1999, pp. 1–40.

8. T.V. Lakshman and D. Stiliadis, “High-SpeedPolicy-Based Packet Forwarding Using Effi-cient Multidimensional Range Matching,”Proc. ACM Sigcomm, ACM, N.Y., 1998, pp. 191–202.

9. V. Srinivasan et al., “Scalable Level 4 Switch-ing and Fast Firewall Processing,” Proc.ACM Sigcomm, 1998, pp. 203–214.

10. P. Gupta and N. McKeown, “ClassifyingPackets Using Hierarchical Intelligent Cut-tings,” IEEE Micro, vol. 20, no. 1, Jan. 2000,pp. 34-41.

11. P. Gupta and N. McKeown, “Packet Classi-fication on Multiple Fields,” Proc. ACM Sig-comm, ACM, N.Y., 1999, pp. 147–160.

12. M.M. Buddhikot, S. Suri, and M. Waldvogel,“Space Decomposition Techniques for FastLayer-4 Switching,” Protocols for HighSpeed Networks, vol. 66, no. 6, IFIP, Lax-enburg, Austria/IEEE Computer Soc., LosAlamitos, Calif., Aug. 1999, pp. 277–283.

13. V. Srinivasan, G. Varghese, and S. Suri, “FastPacket Classification Using Tuple SpaceSearch,” Proc. ACM Sigcomm, ACM, N.Y.,1999, pp. 135–46.

14. M. Kobayashi, T. Murase, and A. Kuriyama,“A Longest Prefix Match Search Engine forMultigigabit IP Processing,” Proc. Int’l Conf.on Communications (ICC 2000), IEEE Press,Piscataway, N.J., 2000, p. 1360.

15. C. Labovitz, G. R. Malan, and F. Jahanian,“Internet Routing Instability,” IEEE/ACMTrans. Networking, vol. 6, no. 5, Oct. 1999,pp. 515–28.

Devavrat Shah is working toward his PhDdegree in the Computer Science Departmentat Stanford University. His interests include

algorithms for networks, probabilistic analy-sis of algorithms and queuing theory. Shahreceived a BTech degree in computer sciencefrom IIT-Bombay.

Pankaj Gupta is a PhD candidate in the Com-puter Science Department, Stanford Univer-sity. His interests include packet classificationand routing lookup for high-speed networks.Gupta received a BTech degree from IIT-Delhi and a PhD degree from Stanford Uni-versity, both in computer science.

Send questions and comments about thisarticle to Devavrat Shah, 268 Packard-EEBldg., 350 Serra Mall, Stanford University,Stanford, CA 94305; devavrat@stanford.edu.

47JANUARY–FEBRUARY 2001

Call for ArticlesIEEE Micro seeks general-interest submissions for publication in

upcoming 2001 issues. These works should discuss the design, per-formance, or application of microcomputer and microprocessor sys-tems. Of special interest are articles on embedded systems.

Summaries of work in progress and descriptions of recently com-pleted works are most welcome, as are tutorials.

Send a 150-word abstract to IEEE Micro’s Magazine Assistantat micro-ma@computer.org. Include your full contact informa-tion (author name(s), postal/e-mail addresses, and phone/faxnumbers). Micro does not accept previously published material.

Check Micro’s home page at http://computer.org/micro forauthor guidelines and work/figure/reference limits. All submis-sions pass through a peer-review process consistent with other pro-fessional-level technical publications, and editing for clarity,readability, and conciseness.

Switching Technology