inside - usenix.orgthe magazine of usenix & sage april 2001 • volume 26 • number 2 {inside:...

THE MAGAZINE OF USENIX amp SAGEApril 2001 bull volume 26 bull number 2

insideCONFERENCE REPORTS

14th Systems Administration

Conference (LISA 2000)

The Advanced Computing Systems Association amp

The System Administrators Guild

amp

4 Vol 26 No 2 login

OUR THANKS TO THE SUMMARIZERS

Lee AmatangeloBrian BaggettDave BianchiPaul Federighi Jim FlanaganSteve HansonDave HomoneyTony KatzBrendan KelliherBrian KirouacVinod KuttyEric LakinDave McFerrenJohn OuelletteSocrates PichardoNicholas SchrieberSam ShafferJosh SimonDoug StewartTheo van DinterCraig VershonSteve Wormley

Photographs taken at LISA 2000 can be

found at

lthttpwwwusenixorgpublicationslibraryproceedingslisa2000confpixpixindexhtmlgt

conference reportsLumeta Corporation Both papers had astudent co-author

Dan Geer USENIX president announcedthe proposed split of USENIX and SAGEcovered elsewhere in this issue BarbDijker SAGE president echoed Danrsquosannouncement reiterated the desire formember feedback and announced the2000 SAGE Award for outstandingachievement Celeste Stokely for herwork in collecting and distributing sys-tems administration information for over10 years

KEYNOTE ADDRESS

THE WORLDWIDE SYNDICATE

JD ldquoIlliadrdquo Frazer User Friendly

Summarized by Jim Flanagan

The artist behind the immensely popularWeb cartoon strip ldquoUser Friendlyrdquo(lthttpwwwuserfriendlycomgt) relatedhow he ventured into self-syndicating hiswork and what he has learned about thetraditional syndication model along theway ldquoUser Friendlyrdquo recently celebratedits third anniversary and has experiencedexplosive growth in Web page impres-sions

Cartoonists are curious about humannature and the traditional syndicationprocess causes these people no end ofpain by imposing on them a fundamentalseparation from their audience which istreated as a market rather than a com-munity Each layer between cartoonistand audience removes revenue alongwith artistic control over content JDtold how Berke Brethed stopped doinghis popular comic ldquoBloom Countyrdquo notbecause he ran out of material or burnedout but because some of the contentoffended Donald Trump who in turnbought the syndicate and sat on the strip

The syndicates can only do what they dobecause they control access to the mar-ket The Web notorious for its lack ofcontrol allows cartoonists to ldquosyndicaterdquotheir own work retain creative controland be more responsive to current

14th Systems Administration Conference (LISA 2000)NEW ORLEANS LOUISIANADECEMBER 3ndash8 2000ANNOUNCEMENTS

Summarized by Josh Simon

The first session started with the tradi-tional announcements from the programchairs Phil Scarr and Reacutemy Evard Philbegan with the following

There are over 1800 attendees mak-ing this our biggest LISA conferenceever

There were 85 papers submitted weaccepted 36 (42)

The Proceedings ran 378 pages Over 1800 messages were sent to

and from the program chairs to planthe conference

Sixty-three percent of the programcommittee changed jobs betweenLISA 1999 and LISA 2000

Thanks go to the program committeeinvited talks chairs network track coor-dinators security track coordinatorsguru coordinator WIPs coordinator thepaper readers Ellie Young and theUSENIX staff Rob Kolstad for his workin typesetting the Proceedings LyndaMcGinley for the terminal room and allthe vendors mentioned in the conferencedirectory

Reacutemy Evard then announced the bestpaper awards

System Honorable MentionmdashldquoDeployme Tellmersquos Package Manage-ment and Deploymentrdquo by Kyle Oppen-heim and Patrick McCormick of TellmeNetworks

Best Papers (tie)mdashldquoPeep (The NetworkAuralizer) Monitoring Your Networkwith Soundrdquo by Michael Gilfix and AlvaCouch of Tufts University and ldquoTracingAnonymous Packets to Their Approxi-mate Sourcerdquo by Hal Burch at Carnegie-Mellon University and Bill Cheswick of

C

ON

FERE

NC

ERE

PORT

Sevents free from the cumbersome reviewprocess Not that all cartoonists need dois sporadically draw cartoons and postthem on the Web They have to beresponsible for all of those things that thesyndicate provides creative and legalsupport and a revenue model

By keeping artistic control the cartoonistgains direct access to an audience ndash acommunity rather than a market ndash andlots of interesting feedback But thiscommunity also needs to be nurturedlest things get out of control For exam-ple an April Foolrsquos joke in which ldquoUserFriendlyrdquo claimed to have received acease-and-desist letter from ldquoa largeunnamed software companyrdquo (which wasreported as fact on Slashdot by co-conspirator Rob ldquoCmdr Tacordquo Malda)resulted in several higher-ups atMicrosoft asking JD who in Microsofthad sent the letter Apparently the read-ership had simply assumed Microsoftwas the culprit and flooded the phoneswith complaints threats and pledges toattack the companyrsquos IT infrastructure onIlliadrsquos signal

The audience also learned a little bitabout what happens behind the scenes atldquoUser Friendlyrdquo in that there is a largishorganization which provides creativeassistance fiscal management and otheroverhead tasks forever dispelling myimage of JD penning the strips alone inhis basement at night and uploadingthem to the Web

Questioners from the audience wanted toknow if the UF crew had any plans tohelp other cartoonists get off the groundwith self-syndication and if UF had anyintentions of becoming a syndicatethemselves UF is waiting for the rightbusiness model before they start involv-ing other artists and they would neverthemselves become a syndicate because itmakes little sense to start placing layersbetween the artist and the audienceThere are other Web-based cartoonistcollectives getting started as well Finally

5April 2001 login LISA 2000

we learned that JD also likes the Webcartoons ldquoGoatsrdquo by Jonathan Rosenberg(lthttpwwwgoatscomgt) ldquoSluggyrdquo byPete Abrams (lthttpwwwsluggycomgt)and ldquoSinFestrdquo by Tatsuya Isheda(lthttpwwwsinfestnetgt)

INVITED TALKS

HOW NOT TO GET FLEECED WITH EMPLOYEE

STOCK OPTIONS

Jon Rochlis The Rochlis Group Inc

Summarized by Theo van Dinter

Rochlis is not a professional financialplanner but he has a strong interest inthe topic and is expecting a degree infinancial planning by the end of 2000The slides from LISA and a goodamount of other option-related informa-tion can be found at lthttpwwwrochliscomoptionsgt

This talk covered a lot of ground What isstock What are stock options What arethe tax rules Is a given stock option offerany good When should the options beexercised Unfortunately Rochlis onlygot to cover about half of the informa-tion in the slides before running out oftime There is a lot of good informationincluding financial planning-relatedlinks available on the Web site listedabove

The overall conclusions from the talkwere

Try to get incentive stock options(ISOs) whenever possible

Stock option value is a percentage ofcompany market capitalization notdirectly related to the number ofshares

Donrsquot be emotionally attached tooptionsshares Do research and sellif it looks like the stock will losevalue Shares can always be boughtagain at a lower price

Be aggressive with tax deductionsDonrsquot lie but claim everything possible

SAGE UPDATE

Barb Dijker SAGE President

Summarized by Lee Amatangelo

Dijker provided a status report on SAGEfor the past year projects in progress andfuture plans

The largest topic looming over the entireLISA 2000 Conference starting with astatement during the Opening Sessionfrom USENIX President Dan Geer is thattalks are underway to determine if thebest course of action for both USENIXand SAGE is to have SAGE become a sep-arate entity

Perhaps the second biggest issue dis-cussed during this update and at otherSAGE meetings during the conferencewas SAGErsquos Certification Project under-taken as a way to elevate systemnetworkadministration to the status of a profes-sion But getting there is going to taketime Along those lines SAGE has alreadydone a fair amount toward defining thevarious levels of systemnetworkdata-base administration work

Several student internship programs andinterns present at the LISA 2000 Confer-ences were recognized The SAGE boardis very much in favor of supporting andpromoting more internships and theaudience was encouraged to create newinternship programs for systemnetworkadministrators

There was also a call put out for mentorsSAGE encourages additional seniormembers of our profession to help othersgrow in this profession in the spirit of thewhole open source mentality Thoseinterested in mentoring or being men-tored were asked to contact SAGE

Ideas were presented and solicited onhow SAGE can market itself Marketingwill now become even more important ifSAGE is to become a separate entity

Other major highpoints of the updateincluded the SAGE salary survey resultsThe results are displayed in many ways

based on various criteria Salaries can belooked at based on gender years of expe-rience geographical location educationallevel certifications achieved and numberof systems (and operating systems) sup-ported to name the biggies

Following the formal update presenta-tion the floor was opened for questionsCollective Technologiesrsquo Jeff Tyler saidthat he had not heard a compelling rea-son from either side as to whether SAGEshould stay with USENIX or break off asa separate entity One response from theboard and other members of the audi-ence was that SAGE and USENIX havetwo very different charters Howeverregardless of whether the split happensor not the board members of bothUSENIX and SAGE assured everyonethat there would still be ventures andevents put on jointly by USENIX andSAGE and that a great working relation-ship between the two groups would bemaintained

THE DIGITAL HOUSE

Lorette Cheswick


Lorette Cheswick along with her not-so-famous husband Bill (I think he mayhave worked at Bell Labs once) and a lit-tle help from their two children Kestreland Terence presented their ldquoDigitalHouserdquo was very interesting entertain-ing and highly educational talk

The Cheswicks have a talking houseWhenever the doorbell rings a messagegoes out over the intercom system stat-ing ldquoSomeone is at the doorrdquo This mes-sage is also heard by the visitor and isparticularly amusing to younger chil-dren

Other ideas soon followed Wouldnrsquot it benice to know when the mail had arrivedSo a wireless motion detector is posi-tioned in the back of the curbside mail-box Whenever the mailbox door opensand shuts a message goes out over theintercom system ldquoHoney yoursquove got

6 Vol 26 No 2 login

mailrdquo (which was quickly changed dueto its political incorrectness)

The intercom system also announcesdaily astronomical events such as thetimes for sunrise sunset iridium flarescomets planet risings and settings etc Inaddition the intercom provides stockquotes throughout the day

Here are more details from their presen-tation

The Cheswick digital house includes

Home Ethernet and wireless Platform for firewall-free experi-

ments Caller ID to serial port X10 Linux sound card to home

intercom

The Cheswicks have 11 computers ontheir home LAN right now

The full Powerpoint presentation islocated at lthttpwwwcheswickcomgtthen select ldquoPowerpoint slides forLorettersquos Digital House presentation atthe New Orleans LISA 2000rdquo Or simplyselect lthttpwwwcheswickcompptdigitalpptgt

EXPERIENCES WITH INCIDENT RESPONSE AT

OHIO STATE UNIVERSITY

Steve Romig OSU-IRT

Summarized by Brendan Kelliher

Before the foundation of the IncidentResponse Team at Ohio State UniversityOSUrsquos computer security was conductedon an ad-hoc part-time basis In early1996 Steve Romig a campus systemadministrator was asked to be part of afull time professional systems securityforce which would become the OSU Inci-dent Response Team

In the summer of 1996 the OSU-IRTresponded to a local ISPrsquos complaintsabout attacks emanating from campussystems A local hacker group the LoTecswas illegally accessing university systems

and using them as launch points toattack other sites

Steve traced the hackers back to the cam-pus modem pool He had to work withthe local phone company (AmeriTech)which caused him much aggravationWhen tracing a call in real time hewould be passed from operator to engi-neer each trying to complete a separatepart of the trace

Some of the hackerrsquos login sessions lastedfor days One lesson he learned is thatphone traces can be done ldquoafter the factrdquoThe phone company has logs of all callsand can do traces going back for monthsThis freed him up to concentrate on whatsystems and accounts the intruders werecompromising

By watching the logins of known hackedaccounts he noticed a pattern emerge intheir sign-onsign-off times Just after thelocal schools let out he would see the firstlogins Then around dinnertime hewould see them logout for a brief periodthen sign back on shortly after Hisassumption that the intruders were localhigh school kids would later turn out tobe correct

Steve attended hacker meetings alongwith a local undercover police officerThey learned the identities of several ofthe hacker group members one of whomhad a history of traffic violations

In October of 1996 the hackers started toattack ldquomilrdquo sites which got the attentionof the FBI Steversquos information was usedto build the government case One inter-esting technique the hackers used was topersonally respond to complaints from avictim they had hacked In September of1997 nine members of the LoTecs wereserved with warrants

Here are some recommended steps youcan take to guard against breaches insecurity

C

ON

FERE

NC

ERE

PORT

S

Determine what measures yourcompany is willing to take againsthackers

Create an Incident Response Team Log everything Make contact with your local

authorities many police depart-ments have established ldquowhite col-larrdquo crime units which also covercomputer crimes

Work out security procedures andemergency contacts with your telcoor Internet access provider

Never audit your own security Be patient when dealing with LEOs

(law enforcement officers) Document all your steps when track-

ing an intruder your notes will becrucial if the case ever goes to court

INTEGRATING LDAP INTO A HETEROGENEOUS

ENVIRONMENT

Leif Hedstrom Netscape


Hedstrom started this talk with a briefoverview of what LDAP is and is notLDAP is a client-server protocol with itsroots in the OSI directory standardX500 LDAP is not a database but merelythe protocol for transmitting and format-ting directory services information onthe network An LDAP entry consists ofone or more attribute-value pairs andcan have multiple values associated witha single attribute The objectclass attri-bute defines what attributes an entry maylegally have and provides a limited objectinheritance The standard schemas whichship with most LDAP directory serverscan be extended but you should try tostick with the supplied schemas ifpossible

Each entry has a globally unique identi-fier called the Distinguished Name orDN which is a path along a tree structurecontaining nodes like country organiza-tion department and name The DN ispublic information so it should not con-tain privileged information such as a per-sonrsquos SSN A Relative DN (RDN) is a


shorter part of the DN which is locallyunique (such as within a company) anexample is a userid or uid When design-ing a Directory Information Tree (DIT)try to keep it as flat as is reasonablebecause later changes will be easier toaccommodate

LDAP will make your life easier but youwill have to treat it like any other enter-prise infrastructure element and make itrobust LDAP can be used to integrateseveral information sources (HR facili-ties NIS NT Domain email mailinglists) but you will need to get buy-infrom all the interested parties

One advantage to implementing LDAP isthat you can assign ownership of differ-ent slices of the data to various groupsand delegate management of the data tothose groups This reduces load on thehelp desk Different LDAP vendorsimplementations handle access controllists (ACLs) differently (as it is not part ofthe LDAP spec) The iPlanet LDAPserver for example provides very power-ful ACL mechanism and default behav-iors (such as automatically giving theowner of a mailing list ldquoownershiprdquo tothat entry)

LDAP can be used to replace or back endNIS for UNIX user information Solaris 8supports LDAP in the etcnsswitch fileout of the box There are scripts availableto migrate your NIS databases to LDAPand the schema is described in RFC 2307On UNIX implementations which sup-port Pluggable Authentication Modules(PAM) there are modules which allowdirect LDAP binding over SSL (no pass-words over the Net in the clear)

When you have disparate data sourcesyou may want to investigate one of thecommercially available LDAP metadirec-tory solutions These allow for bi-direc-tional synchronization of data frommultiple LDAP directories and otherlegacy sources (via ldquoconnectorsrdquo) auto-matically and are designed to resolvenamespace conflicts Depending on the

complexity of your problem you mightbe able to build your own metadirectorybut most of the time you will have to buyone There are not currently any opensource metadirectory systems Most commercial metadirectories are extensi-ble allowing you to write your own connectors

Another approach is to gateway legacydata sources into LDAP A commonexample of this is ypldapd which allowsyou to store your NIS maps in LDAP butuses traditional tools and clients to accessit This tactic should only be used as atransitional tool rather than a solution

The change log is used for server replica-tion and can be a nice hook into yourdirectory server to accomplish somemetadirectory functionality The changelog is data in the LDAP directory itselfand is protected by ACLs but in an all-or-nothing fashion giving a possibleexposure of data that is otherwise pro-tected with more granularity in the direc-tory Keep the change log protected Thechange log can also be used for disasterrecovery if you back it up

Exporting LDAP data can accomplish aldquopoor-manrsquos metadirectoryrdquo You can usescripts to massage LDAP-exported datainto NIS maps DNS zone files or what-ever Itrsquos easy and itrsquos fun

NT presents special difficulties to LDAPdeployment especially because ActiveDirectory wants to be in controlMicrosoft also made some poor decisionsregarding RDN and uses a proprietarypassword encryption scheme forcing theuse of plaintext passwords Having a sin-gle namespace for UNIX and NT userswill help you avoid various problems

MAPPING CORPORATE INTRANETS AND

INTERNETS

Bill Cheswick Lumeta Corporation

Summarized by Steve Hanson

Many of us often wonder what it wouldbe like to leave our safe jobs in the corpo-

rate world to start a company Part of BillCheswickrsquos talk addressed this since hehas recently spun off Lumeta fromLucent starting a company to map cor-porate intranets as a service

Most of the talk however was about thework Lumeta is doing Some of this workdeveloped out of infrastructure protec-tion done for the government Mostintranets are completely out of controland nobody really knows anymore whatis on their networks Some preliminarywork is being done on mapping intranetsand making simple visualizations of thesystems Lumetarsquos work was compared tosome other projects in these areas suchas MIDS and CAIDA Different visualrepresentations of the intranets and theInternet have been done Lumeta also didsome work for the government duringthe Yugoslavian crisis mapping the Inter-net in Yugoslavia which gave good indi-cations of what portions of the countryhad had their power knocked out bybombing

One of the most interesting aspects of thetalk was the maps themselves Some ofthem are quite beautiful and some havebeen purchased by museums as artworksOf course this information is being usedprimarily in the research communityand Lumeta hopes a large commercialmarket for their services will develop

Maps lthttpwwwpeacockmapscomgtInformation lthttpwwwlumetacomgt

THE DESIGN AND IMPLEMENTATION OF HIGHLY

SCALABLE EMAIL SYSTEMS

Brad Knowles Belgacom Skynet SANV

Summarized by Brian Baggett

Knowles former email admin for AOLgave a great talk on the problems faced indeveloping and running large-scale emailsystems His talk focused less on imple-mentation and more on fundamentalsand architecture

The approach taken by academia is oftendifferent from that of commercial insti-tutions Academia often reinvents the

8 Vol 26 No 2 login

wheel and does things on a small scalebut occasionally it does something revo-lutionary By contrast the commercialworld has no problem buying solutionsthe time it takes to bring a product tomarket is crucial and so the processtends to be more evolutionary than revo-lutionary Most of their revolutionsfocused on scaling

The underlying problems with all of thepotential solutions are that none of themscale to handle 1 million-plus users ontheir own Eliminating single points offailure or getting away from inefficienttechnologies like NFS has proven too dif-ficult for many Knowles summarized thepros and cons of POP3 and IMAP andidentified the big bottleneck that ham-pers mail server scaling (IO) This was ahighly informative talk chock full ofinteresting facts and data the results ofwhich can be found athttpwwwusenixorgeventslisa2000invitedtalksknowlespdf

SANS mdash FROM A TO REALITY

W Curtis Preston Collective

Technologies


SAN systems are becoming a fact of lifein most production environments but itis not always clear if the SAN systemsbeing purchased are serving a purpose orare just being bought because they arethe new and hot technology

Prestonrsquos presentation on SAN systemsdiscussed the different technologiesinvolved in SANs and how to decide ifSAN technology is a good fit for yourneeds

He described the three competing dis-tributed storage technologies ndash WANNAS and SAN and then went on to dis-cuss the different hardware configura-tions available for SAN The advantagesof SAN (easy allocation sharing backupetc) were covered as was its cost effi-ciency

Finally this talk discussed some of thecurrent pitfalls of SAN (need to test thor-oughly incompatibility between different

hardware) and how to decide if SAN isfor you

WHY THE DOCUMENTATION SUCKS AND

WHAT YOU CAN DO ABOUT IT

Steven Levine SGI


Levine spoke and sang about documen-tation Steven a technical writer talkedabout four major subjects myths diffi-culties projects and improvements

First Steven talked about some mythsOne is that writers are editors In realitywriters not only edit stuff others (such asdevelopers) write but write original con-tent maintain other existing documentsand produce both hardcopy and onlinehelp and Web-based documentationThey also coordinate and organize andare detectives they have multiple infor-mation sources and work with variousgroups or departments They are alsoresponsible for documentation consis-tency and legal issues

He then discussed some of the difficultiesthat writers face He started this sectionwith a song and yes all 300-plus audi-ence members were singing along withthe chorus The major difficulties are lackof resources conflicting perspectiveslittle (if any) usability testing of the doc-umentation shorter release cycles dis-tinctions in hardware and software theproblems of writing from experience onas-yet-nonexistent products and havingto rely on developersrsquo time and interest toimprove the documentation

Third he discussed some of the typicalprojects that writers are involved in Theyare responsible for not only administra-tive documentation but also online docu-ments procedures and examples andman pages (which may not be sexy butare certainly very useful)

Fourth Steven discussed how to improvethe documentation The short answer isitrsquos a two-way street If you see somethingneeding work in a document let theauthor know Therersquos always some form

C

ON

FERE

NC

ERE

PORT

Sof contact information (even if itrsquos postalmail) Document what you want solvedDocument what you did to work arounda problem to help others not have to gothrough it themselves Formalize yourinformal people networks if yoursquore adeveloper take your writer to lunch Pro-duce libraries of examples procedurestricks you use and so on Collaboratewithin the company across departmentlines Collaborate with friends andacquaintances at other companies

REFEREED PAPERS

SESSION DEEP THOUGHTS

Summarized by Socrates Pichardo

THEORETICAL SYSTEM ADMINISTRATION

Mark Burgess Oslo College

Mark attempts to demonstrate that sys-tem administration can be modeled incertain instances and that this theoreticalmodel can be built and used to furtherunderstand system administration vari-ables and their interdependencies This isa very important concept once you canidentify all relevant parameters andmodel a particular problem you can pro-ceed to optimize these variables toachieve maximum results

Mark started his presentation by defininga simplified system administration modelin which users OSes resources policiesand states are the main components

Markrsquos group has concluded that systemadministration problems fall into thesecategories

Random behaviors or type I mod-els Variables follow random patternswithin a well-defined cause-effectrelation Things like averagesmedian and statistical theory can beused to optimize these models Moststability problems will follow thisbehavior

Anthropological behaviors or typeII models Variables follow anthro-pological behaviors within certainboundaries Game theory andhuman conduct analysis can be used


to optimize these models Most uti-lization problems will follow thisbehavior

In other words influences on the systemscan thus be classified as either randomstochastic or passive (type I) or as inten-tional adversarial or strategic (type II)depending on the significance of thechange

AN EXPECTANT CHAT ABOUT SCRIPT

MATURITY

Alva L Couch Tufts University

Couch presents his solution to currentscripting tools and language limitationson solving complex system managementtasks To circumvent current limitationshe is developing an ldquointerpreterrdquo calledBabble which will take XML-like direc-tives and mark up tags and interpretthem into desired configuration tasks

To accomplish this he has created hisown set of directives called Stream-Struc-ture Markup Language or SSML basedon the ldquoJackson System Designrdquo from thepunch card era Jacksonrsquos Principleclaimed that the way to properly design aprogram for processing punched cardstacks is to link the structure of the pro-gram with the structure of the stack thatit processes Alva expanded this principleinto ldquoThe structure of a fully functionalinteractive script is exactly parallel to thebranching and looping structure of thedevice interactions in which it mustengagerdquo

SSML and Babble introduce a new levelof instrumentation that will help systemadministrators tackle complex adminis-tration tasks but at the expense ofextremely limited functionality and lowreusability Right now there are only lim-ited sets of complex tasks that Babble canaddress with success and SSML scriptsare very dependent on a particular revi-sion version of the devices being man-aged Each device revision requires acompletely independent Babble script

AN IMPROVED APPROACH FOR GENERATING

CONFIGURATION FILES FROM A DATABASE

Jon Finke Rensselaer Polytechnic

Institute

Much of Rensselaerrsquos site configurationinformation is stored in a relational data-base In the past lots of little custom Cprograms and scripts were needed toextract this information in the appropri-ated format for server and daemons Thisapproach proved to be difficult to main-tain and expand

Maintenance was cumbersome due to thevariety of scripts techniques and pro-gramming styles found at the site Expan-sion into new operating systems wastime-consuming since all these scriptsneeded to be ldquoportedrdquo to new hardwareand operating system revisions

Now Rensselaer is using a centralizedapproach to this problem They havegathered all logic and intelligence neededfor configuration file generation intotheir Oracle database by using PLSQL as their scripting tool They generate alltheir configuration files in the databaseitself then write a short program todump the ldquofilesrdquo out of the database andinto the file system of the target machineThis allows all of the file generation codeto be stored and maintained in the cen-tral database using a consistent set oftools In addition the program to copyfiles could be generic and once built for aparticular operating environment couldbe used for any of the files they mightgenerate for that system

Rensselaerrsquos solution proves that central-ization when done right can have a sig-nificant positive impact on the overallmanageability of an IT infrastructureTheir usage of commercial tools (ieOracle PLSQL) makes their tools andscripts a bit more difficult to implementAt the end of his presentation Jon madea ldquocall for helprdquo for anyone interested inldquoportingrdquo their scripts into any of theopen source products (ie MySQL)

SESSION YOU A ROCK AND A HARD

PLACE

Summarized by Craig Vershon

FOKSTRAUT AND SAMBA mdash DEALING WITH

AUTHENTICATION AND PERFORMANCE ISSUES

ON A LARGE-SCALE SAMBA SERVICE

Robert Beck and Steve Holstead

University of Alberta

Robert and Steve noticed a performanceproblem with the Samba server that wasbeing used as a gateway to AFS The sys-tem was getting an unanticipated newload that couldnrsquot handle all the usersrsquoauthentication

The Samba server was running as an AFSclient gateway to things like Windowsclients They found they had to runSamba with clear text passwords enabledwith password crack but found an issuewith some Windows clients sending pass-words in all caps This would allow themto authenticate to Samba but not to Ker-beros which requires more varied pass-words

Once they implemented the server theyfound it to be highly CPU-bound Theserver was receiving repeated passwordfailures They found a pattern threebogus attempts then the real passwordwas sent Windows was sending the ldquowin-dowsrdquo password instead

The solution FOKSTRAUT patches forSamba to make a DBM password cacheFirst it caches the password that failed Itstores this in a database and keeps a fail-ure count After three failures it checksagain and resets to zero after successThen they cached the ldquocorrected caserdquosuccess This was stored in a clear textdatabase They found this ldquoevilrdquo andunsecure but a compromise had to bemade somewhere

Available atltftpsunsiteualbertacapublocalpeoplebeckfokstrautgt

10 Vol 26 No 2 login

IMPROVING AVAILABILITY IN VERITAS ENVIRONMENTS

Karl Larson Tellme Networks Todd

Stansell Certainty Solutions

Karl and Todd spoke of problems andsolutions they found in their VERITASimplementations

Some of the tools they created or used

vxstat2gnuplot this program con-verts the output of vxstat into a use-able format for gnuplot I thoughtthis was really useful to see a graphicversion of the diskvolume usage

cricket used for trends-based moni-toringlthttpcricketsourceforgenetgt

save-vxlayout this script saves VMconfig details good in use for disas-ter recovery info

synch from EMC retrieves Sym-metrics internal configurationdetails

emcprints shows all back-end con-trollers devices etc adds this infoto the vxprint output

They found problems with millions ofsmall files on a single file system Themetadata becomes a performance bottle-neck with VxFS It uses metadata forintent logs (journaling) Changing mil-lions of files causes changes to muchmore than just the files themselves Bydefault all the metadata is stored at thebeginning of the file system The spacereserved could be too small They foundthat by using Quicklog they could moveand store the metadata on a separatedevice

Running backups on millions of sequen-tial files daily makes it hard to obtainconsistent ldquopoint in timerdquo backups Theycame up with two fixes Volume Managersnapshots and using file system snap-shots

Cool tips and tricks for VERITASlthttpwwwvxideasorggt

DESIGNING A DATA CENTER INSTRUMENTATION

SYSTEM

Robert Drzyzgula Federal Reserve

Board

Drzyzgula outlined the context of hisproblems

Several conflicting production cycles Tight deadlines Enormous economic models Highly variable capacity require-

ments Capacity is higher priority than

reliability Drzyzgula seems to have a limited budgetwith which to work They purchase manyparts in bulk and assemble systems them-selves He has been working on designingand building a monitoring and control-ling device to be used on all his varioussystems in the data center The goals wereto be able to monitor such things aspower usage and temperatures and to beable to control power cycles consoleaccess etc He spoke about the variouschips sensors and other hardware hewas using to attempt to implement thisHe has been able to get a small workingprototype to work in a limited environ-ment

SESSION USERS AND PASSWORDS

AND SCRIPTS OH MY

Summarized by Sam Shaffer

USER-CENTRIC ACCOUNT MANAGEMENT WITH

HETEROGENEOUS PASSWORD CHANGING

Douglas Hughes Auburn University

Hughes details development of a Web-based tool to allow students to changetheir UNIX andor NT passwords (NTauthentication is via Samba) The ldquoUser-Centricrdquo in the title indicates that the sys-tem was developed with inexperiencedstudents in mind

The paper lists five similar systems andtheir principal parameters Douglas indi-cated that a lot of information whichmight have made the other systems valu-able was not available to him He is look-ing for someone to maintain the code

C

ON

FERE

NC

ERE

PORT

Swhich is available atlthttpwwwengauburnedu~dougsecondhtmlgt

PELENDUR STEWARD OF THE SYSADMIN

Matt Curtin Interhack Corp Sandy

Farrar and Tami King Ohio State

University

This paper documents what seems to bea rather thoroughly automated accountmanagement system in use at Ohio StateUniversity

Bottom line this system modifies studentand other accounts in response tochanges in the university class databaseImplementing it has greatly reduced theamount of time required to add anddelete about 15000 accounts per schoolterm The system is not available to thepublic because it isnrsquot ready for primetime As such it is effectively ldquoyet anotherset of design guidelines for an accountautomation toolrdquo and provides somespecifics of the implementation thatmight allow someone else to design andimplement a similar system again

NETWORK INFORMATION MANAGEMENT AND

DISTRIBUTION IN A HETEROGENEOUS AND

DECENTRALIZED ENTERPRISE ENVIRONMENT

Alexander D Kent and James R Clif-

ford Los Alamos National Laboratory

The paper presents another unique set ofcircumstances which had to be incorpo-rated into a user-friendly tool to allowpeople to manage their own data (suchitems as email aliases and email serverpasswords) There are several interestingcomponents of this system though Oneis that changes to the database causenotification to an ldquoevent trigger daemonrdquowhich uses inetd to induce an update toan LDAP server

The system source may be available Thepaper indicates that US encryptionexport issues require that the source becontrolled Make requests to the authorsfor information on requirements


SESSION THE TOOLSHED


XPS DYNAMIC PROCESS TREE WATCHING

UNDER X

Rocky Bernstein Breakaway Solutions

In a talk as dynamic as the topic matterRocky described xps which provides aview of the process table laid out as a treerooted at the init with colors distin-guishing processes by owner or by state(running or waiting on IO) Whatprocesses are shown can be determinedby user-specified filters and clicking onthe process names can run a user-speci-fied program such as ps to get specificinformation or lsof to get the files openby that process

The current version of xps is writtenusing the Motif toolkit but work is inprogress to port it to GNOME Much ofthe talk was spent weighing the variousvirtues and drawbacks of GNOME versusMotif and how the GNOME view of theworld changes how the xps problem isattacked and how one goes about opti-mizing for efficiency GNOME also hastools which make life easier such asGlade which builds dialog boxes that aremuch prettier than Motif rsquos

Rocky then demonstrated an instance ofwhere xps might give more insight thantraditional tools like ps or top He wentto a source directory with a fairly com-plex make procedure and typed makeThe make process showed up in the xpsdisplay and alongside (or ldquounderrdquo) thatyou could see all the subsidiary awk com-mands and other commands beingspawned to do the work and disappear-ing dynamically

One questioner asked about the poll-based nature of xps having to readthrough the entire process table eachrefresh and if it would be more efficientto somehow detect or trap calls to forkand vfork Rocky said that it wasnrsquot clearhow to go about that and that if youcould you might introduce problems

with modifying the state of a programrather than simply monitoring it

EXTENDING UNIX SYSTEM LOGGING WITH

SHARP

Matthew Bing and Carl Erickson Grand

Valley State University

As grad student admins of 30 machinesthe authors were being overwhelmed byhaving to try to winnow the interestingdata from the volume of syslog data thatwas being generated After giving a briefoverview of how syslog works Matt wenton to present a list of areas where syslogcould be improved

The routing of syslog messagesdepends only on their prioritywhich is the combination of facilityplus a level which is not very flexi-ble You canrsquot add facilities to syslog

There is no standard message struc-ture after the timestamp and pid

The priority information is not writ-ten to the logfile and so is lost

In a centralized logging architecturethe timestamps are those of the orig-inating system and if the clocks onthose systems are not in sync eventcorrelation is not feasible

It is not possible to detect if the log-file has been tampered with

UDP Say no more

Log-watching systems like swatch or log-watch donrsquot completely compensate forthese defects in that they donrsquot providerealtime analysis or maintain statebetween runs so they canrsquot detect recur-ring problems and change their behavior(like stop paging you for the same prob-lem)

SHARP (Syslog Heuristic Analysis ampResponse Program) is the authorrsquosresponse to these problems Rather thanreplace syslogd SHARP provides a dae-mon (sharpd) which also receives a copyof every syslog message Modules com-piled against the SHARP library can thenregister to get messages of various priori-ties The messages which the modulesreceive are timestamped by sharpd for

event correlation The modules can thendo anything with the message put it in afile alert a user or bounce the messageback to syslog at a different priority

Examples of modules were

Mark expects a message from eachmachine at a certain interval andlogs a high priority message if itdoesnrsquot get it

UserAlert learns about usersrsquo pat-terns of logins (time and location)and notices behavior changes

ProblemAlert after a number ofrepeated messages of a certain prior-ity they will start getting sent backthrough syslog at a higher priority

While SHARP will work with syslog theauthors recommended nsyslog as areplacement to work with SHARP asnsyslog preserves the priority of mes-sages uses TCP and SSL to preventspoofing and uses chained hashing toprevent modifications of the logfile

Planned developments include a Perlinterface for modules access to globalconfiguration across all modules andmaking SHARP completely thread safe

PEEP (THE NETWORK AURALIZER) MONITORING YOUR NETWORK WITH SOUND

Michael Gilfix Tufts University

[Winner of the Best Student Paper Award]

The Peep tool is an experiment in lever-aging innate human skills in distinguish-ing subtle deviations from the normalstate Peep uses a continuous non-intru-sive audio representation of the health ofyour network in terms of events stateand heartbeat types of sounds Thesounds selected for use with Peep are allfrom nature (waterfalls bugs birds etc)since this was thought to be the leastintrusive

Peep is a departure from the currentstate-of-the-art monitoring tools in thatwhere current tools are visual problem-centered and provide negative reinforce-ment Peep is aural normalcy-centered


and provides positive feedback Peepattempts to remain ambient to takeadvantage of unconscious processingand the sounds are all mixed together sothat combinations of sounds become sig-nificant If your network sounds like itsounded yesterday than everything isfine

Peep has a ProducerConsumer architec-ture which supports either distributed orcentralized configuration It is UDP-based uses auto-discovery by both clientsand servers and employs leasing to han-dle servers that go offline

Michael gave a demonstration of PeepFirst he played discrete sounds such asbird chirps which corresponded toincoming and outgoing mail bad DNSlookups and telnet connections Thenthere were some continuous sounds likerunning water and general forest insectnoise which represented load averageand concurrent users respectively Thenhe played a sample of actual Peep outputfor low load which sounded like being inthe forest near a stream After that heplayed a high load average sample andwhile I didnrsquot feel like it was quite time tostart filling sandbags it seemed a littleless comfortable

Someone from the audience commentedthat for rare events you might forgetwhat sound went with what eventMichael said that they were looking for asolution to that such as a GUI quick ref-erence utility Another questioner askedwhat the overhead was on the server sideMemory is really the biggest bottleneckas all sounds are loaded into memoryMost of the processing overhead is in thesound mixing In trials they were onlyable to drive the server load average up to06

SESSION 1984


THRESH ndash A DATA-DIRECTED SNMP THRESHOLD POLLER

John Sellens Certainty Solutions

Thresh is a simple SNMP monitor toolthat lives in between realtime alert moni-toring systems (ie Big Brother) andtrend analysis and history tools (ieCricket) The power of this tool lies in itssimplicity Thresh is an elementary butelegant implementation of SNMP moni-toring services with an emphasis on easyconfiguration low system overheaddecent notification and some basic his-tory and logging facilities

In spite of Threshrsquos low system overheadit has some scalability issues Its mainconstraints are the lack of parallelismconfiguration complexity and notifica-tion throughput

If you need some basic SNMP monitor-ing without all the hassle of configuringand maintaining a feature-rich SNMPconsole then Thresh could be the toolfor you otherwise stick to some of themore capable SNMP consoles availabletoday

EEMU A PRACTICAL TOOL AND LANGUAGE

FOR SYSTEM MONITORING AND EVENT

MANAGEMENT

Jarra Voleynik eEMUconcept Pty Ltd

Voleynik described eEMU as a monitor-ing and management event consoleeEMU is a client-server system that pro-vides for rapid development of monitor-ing agents Beside its console capabilitieseEMU has a scripting language that takesadvantage of heuristic algorithms imple-mented at the server

One of the things that sets this tool apartfrom the market leaders (ie TNGPatrol OpenView and Tivoli) is the wayit handles aggregates and presentsalarms While other console solutionswill rely on color code representationsand multiple windows of informationeEMU uses textual messages for each

C

ON

FERE

NC

ERE

PORT

Sevent in a simple intuitive interface calledeEMU browser By default the eEMUbrowser display only resources in ldquoalarmstaterdquo

For its implementation eEMUconceptLtd has decided to write its own eEMUagents and have them communicatingwith eEMU servers by using the eEMUprotocol eEMU works on the premisethat all status information is handled bythe eEMU server therefore eEMU agentsare simple scripts or programs that usethe emsg program to send messages tothe server

One debatable design characteristic istheir usage of TCP and not UDP forclient-server communication By usingTCP and not transmitting ldquoSystems OKrdquomessages they can avoid the commonldquoUDPrdquo storms generated by SNMP con-soles and their polling efforts On theother hand we can argue that the over-head generated by TCP connections aswell as the lack of ldquoSystems OKrdquo mes-sages could produce some challengingprogramming problems for the develop-ers and minimize their utilization gainsOn their labs they have been able tomonitor 100 systems on a 33Kbps dialupline or 1000 messages a minute on a400MHz Pentium PC

The power of the eEMU messaging lan-guage can be easily illustrated on theeEMU agents eEMU agents with a fewlines of code can handle complex moni-toring scenarios

Finally eEMU has been successfully inte-grated with some of the major monitor-ing software vendors This integrationcan be accomplished by using eEMUaction scripts as well as other scriptinghooks to their event engine

ABERRANT BEHAVIOR DETECTION IN TIME

SERIES FOR NETWORK SERVICE MONITORING

Jake D Brutlag Microsoft WebTV

Realtime monitoring of service networkscan generate vast amounts of time seriesdata Open-source tools like RRDtool


and Cricket can help you with collectingstoring and visualizing this data but forlarge networks you still need a method-ology or tool to help you identify failuresandor abnormal situations

Microsoftrsquos WebTV division was facingthis problem and the amount of databeing generated was enough to distracttheir network administrators from theimportant issues facing their networksTheir solution was to integrate a modelbased on exponential smoothing andHolt-Winters forecasting into theCricketRRDtool architecture

Their model takes into consideration thefollowing characteristics of time seriesdata

A trend over time A seasonal trend of cycle Seasonal variability Gradual evolution of regularities

over time

The Aberrant Behavior Detection modelunpacks into three pieces each buildingon its predecessor

An algorithm for predicting the val-ues of a time series one time stepinto the future

A measure of deviation between thepredicted values and the observedvalues

A mechanism to decide if and whenan observed value or sequence ofobserved values is ldquotoo deviantrdquofrom the predicted value(s)

This model was implemented by enhanc-ing the RRDtool with five new ldquoconsoli-dation functionsrdquo

HWPREDICT an array of forecastcomputed by the Holt-Winters algo-rithm one for each Primary DataPoint (PDP)

SEASONAL an array of seasonalcoefficients with length equal to theseasonal period

DEVPREDICT an array of devia-tion predictions

DEVSEASONAL an array of sea-sonal deviations

FAILURES an array of Booleanindicators

On the Cricket side Cricket 11 alreadyincludes a new type of monitor-thresholdspecific for aberrant behavior detectionCombining these two tools they wereable to monitor and alert on aberrantbehavior conditions

SESSION THE SORCERERrsquoS

APPRENTICE

Summarized by Vinod Kutty

PIKT PROBLEM INFORMANTKILLER TOOL

Robert Osterlund University of Chicago

PIKT is a system configuration manage-ment tool addressing problems that toolssuch as cfengine are designed for but in amore general purpose way It monitorsand warns of system problems and hasfeatures that allow it to take correctiveaction if needed The focus is on manag-ing large numbers of machines ratherthan on individual machines

Sysadmins typically write custom scriptsto address these issues but problems ofOS diversity code robustness and main-tainability specificity to certain tasksscheduling scripts error logging scriptand configuration file distribution andso on plague this approach

PIKT is designed to solve a lot of theseproblems in a fairly platform-indepen-dent (ie UNIX-flavored) manner At itscore is an embedded scripting languageand a configuration file pre-processorthat can be used with languages otherthan the PIKT language It also includes ascheduling system a distribution mecha-nism (like rsyncrdist) and a remoteprocess execution facility (like rshssh)

The typical deployment involves a centralldquomasterrdquo machine which controls ldquoslavesrdquo(ie clients) Configuration files aremanaged on the master then runthrough a tool that pre-processes andinstalls files pushes changes to slaves

executes remote commands and so on Ascheduling daemon on each client runsalarm scripts to monitor various aspectsof a system (eg disk usage runningprocesses etc) and a flexible logging sys-tem is provided There is someclientserver security implemented usingsecret-key host authentication

Some use cases not directly related tomonitoring include installing and man-aging non-PIKT scripts and configura-tion files (eg inetdconf) documentdistribution and managing security (bycomplementing security tools with logfileanalysis security configuration file main-tenance and so on)

Future work includes a full securityaudit a standard library of configurationfiles a rewrite of the PIKT script inter-preter (possibly using embedded Perl oranother language) improved messagerouting and graphical interfaces for thepiktc and alert management compo-nents

RELIEVING THE BURDEN OF SYSTEM ADMINIS-TRATION THROUGH SUPPORT AUTOMATION

Allan Miller and Alex Donnini Hands-

Free Networks

Companies increasingly have to supporta growing population of users with mini-mal application or other technical train-ing This in turn increases the burden onsupport organizations

An automated support system can helpavoid a crisis and improve the scalabilityof the support organization Howeverautomating support is difficult and oftenleads to a ldquomountain of kludgesrdquo that donot exhibit an understanding of theissues Automation is best suited forrepetitive tasks and touches upon allaspects of a system

Traditional user support involves somekind of problemticket system with adatabase back end that stores solutions topreviously encountered problems Ateach step of the support process humanintervention is necessary to clarify end


user symptoms search existing symptom+ resolution databases escalate therequest and so on This is a labor-inten-sive error-prone process and often relieson mental knowledge rather than a data-base

The automated support system underdiscussion uses a software client insteadof the user to detect and report prob-lems A database is used to track symp-toms and resolutions that includeexecutable code (called ldquoscripsrdquo whichare collections of various modules)Thus the software client can automati-cally resolve the problem if there is amatch

The expectation is that about 80 of allproblems can be solved this way with theremaining 20 involving an escalationprocedure In addition there is a wellknown 80-20 rule in support circles thatsuggests the size of the database that cansolve 80 of all problems is expected tobe about 20 of the size of the universeof solutions

Experience with the system so far hasbeen on Windows operating systemswith a Linux port in the works Somebeta sites are using the software andfeedback indicates a remarkable similar-ity in problems encountered and auto-matically solved despite considerabledifferences in the businesses

Additional uses envisioned for the systeminclude automated administration andmaintenance functions security patchdistribution and automated support formobile and embedded systems

FTP MIRROR TRACKER FIRST STEPS TOWARDS

URN

Alexei Novikov and Martin Hamilton

ITEP

The FTP Mirror Tracker packageattempts to decrease the load of FTP traf-fic on WANs while improving perfor-mance for end users It does this bylocalizing FTP accessible files on mirrorsand employing a transparent scheme to

redirect users to the nearest mirror withthe latest most complete copy of all thefiles needed

A robot gathers directory listings fromFTP servers and a summarizer compo-nent parses these and creates MD5digests on a per-directory basis A data-base back end using MySQL keeps trackof FTP Mirror Tracker data and links col-lections to the domains being tracked Adigest exchange compresses and movesthe digests into a Web-accessible area (forother trackers to access) and front-endprograms provide the means for users toquery trackers An ICP (Internet CacheProtocol) server was also written as ameans to allow cache querying by otherWeb caching systems

This comes into play when users are redi-rected to the closest FTP mirror by usingSquid to redirect URLs to the ICP servercomponent for rewriting

Internal support for URNs (UniformResource Names ie a persistent loca-tion-independent naming scheme thatdecouples location from the name of theresource) has been added to FTP MirrorTracker

The system has been put into produc-tion and preliminary results show a rea-sonable cache hit rate but improvementsare expected Some of the functionalityimplemented on top of Squid have beenfolded into Squid itself as of the distribu-tion of version 23

SESSION FULLY AUTOMATIC


DEPLOYME TELLMErsquoS PACKAGE MANAGEMENT

AND DEPLOYMENT SYSTEM

Kyle Oppenheim and Patrick

McCormick Tellme Networks

[Winner of the Best Paper Award]

Deployme is Tellme Networkrsquos solutionto manage the package update life cycleacross a large number of independentlyconfigured hosts It is highly flexible and

C

ON

FERE

NC

ERE

PORT

Shas been extended to handle many differ-ent types of packages These packagesinclude standard UNIX tools local appli-cations Web site content and voice sitecontent Deployme value can be maxi-mized on packages that require fast fre-quent deployment

Deploymersquos mission is to provide a cen-tral system for tracking the entire lifecycle of software packages Its designinggoals are

Support a wide audience Robustness Augment the development process Flexible destinations Efficient use of network bandwidth Quick pushes Seamless activation Rollback Scalability

On the other hand Deployme designersintentionally left out several featureswhile pursuing simplicity and shortertime to market These features are

No local package management No dependencies No fine-grained operations control

Deployme is written entirely in Perl 5and has a simple three-tier architecture

Although Deployme is a well-imple-mented solution to Tellme Networkrsquospackage management problems it lackscertain featuresservices in certain areasThe authors mentioned some ofDeploymersquos shortcomings

It doesnrsquot have the concept of ldquositesrdquoor several machines sharing a physi-cal location

The lack of ldquotransactionsrdquo at thedatabase level makes it difficult toaccurately determine the integrity ofthe data after a system failure

Deploymersquos lack of multicasting sup-port negatively impacts network uti-lization when the same package issent to a large number of servers


The system has no security featuresas of version 1 There is no controlover who what or where

Tellme Network is currently working onsolutions for many of these shortcom-ings

AUTOMATING REQUEST-BASED SOFTWARE

DISTRIBUTION

Christopher Hemmerick Indiana

University

Netdist is a very complete solution forsoftware distribution that was designedfrom the ground up with security modu-larity flexibility and extensibility inmind Netdist provides an automatedmechanism for system administrators torequest and receive software exports withan immediate turnaround The systemprovides a simple user interface secureauthentication and both user- andmachine-based authentication Each ofthese is configurable on a package-by-package basis for flexibility

Netdist is a modular service The userinterface authentication and authoriza-tion are independent of the export proto-col The author is currently distributingvia NFS but adding an additional proto-col is as simple as writing a script to per-form the export and plugging it intoNetdist

Netdist is implemented using Perl 5 andsome modules from CPAN PGP cron(or any other job scheduling service) andan instance of Apache with at leastmod_perl and preferably a module forsecure transactions The NFS export con-trol scripts have been written for Solarisbut could be easily ported to other UNIXflavors

The only shortcomings of this tool are itslack of installation scripts and availabil-ity Netdist is still pre-alpha and a lotmore work is needed in order to easeinstallation Also several of the Perlmodules and scripts do have host- orport-specific information coded intothem Although each of these instances is

documented the authors will attempt toextract all these values into a configura-tion script in the next version

USE OF CFENGINE FOR AUTOMATED MULTI-PLATFORM SOFTWARE AND PATCH

DISTRIBUTION

David Ressman and John Valdes

University of Chicago

The authorrsquos main requirements were tocreate or buy an automatization packagefor software and patches distribution inorder to improve the level of servicesbeing provided to their end users and toliberate their two SAs (authors) fromthese repetitive tasks Some of the impor-tant characteristics of the solution werecost ease of use current developmentand security

Their solution was to ldquogluerdquo with Perlsome of the ldquobest of breedrdquo tools avail-able for the different tasks They tookCfengine (Configuration and SystemManagement tool) NFS (for their filesystem exports) and RPM (Red HatPackage Manager) and used them asbuilding blocks in addition to Perl andmySQL to create the Web interface aswell as the back-end database

The outcome for the software distribu-tion problem is a Web-based front endwhere users can request which softwarepackage they want to install Once theysubmit their request the system willinsert these requests into the databasecreate the necessary exports and offerusers the opportunity to launch the RPMmodule requested On the patch distribu-tion side hosts will check periodicallywith the software depot server for newpatches available for their OSes andarchitectures Once clients find newpatches they will proceed to install themand to report back the exit code of suchinstalls

Future work on this project will involveexpanding its services to include OSupgrades as well as support for otherUNIX flavors

SESSION BUILDING BLOCKS


UNLEASHING THE POWER OF JUMPSTART ANEW TECHNIQUE FOR DISASTER RECOVERYCLONING OR SNAPSHOTTING A SOLARIS

SYSTEM

Lee Amatangelo Collective

Technologies

A production data center requiresprocesses to reduce downtime of criticalservers as far as possible Apart fromhardwaresoftware high-availability solu-tions a disaster recovery plan is a must

This paper describes a system that pro-vides the following for Solaris systems

Bare-metal recovery Creation of a system snapshot on

optical media Cloning of a system Rollout of multiple system clones

A typical application is one in which theroot drive(s) of a server has failed orbeen corrupted by human error or hard-ware failure Traditional backuprestoremethods are not feasible when the OScannot run

One approach is to use bit-level imaging(or ldquoghostingrdquo in the PC world) whichcan be quite fast but not as flexible assomething like JumpStart which is thealternate approach that performs auto-mated Solaris installations

The solution ndash the Capture and RecoveryTool (CART) ndash combines both tech-niques The tool evolved in an environ-ment where security was important andthis is reflected in the requirements

No magnetic media allowed Recovery media must be bootable There should be minimal user inter-

action Multiple sets of removable media

must be handled (eg if a snapshotrsquossize requires three CDs)

Should not involve directory servicessuch as NIS NIS+

Should not depend on NFS


The implementation depends on theSolaris installboot command ndash which canplace boot blocks on optical media ndash andthe customizable nature of JumpStart

A good understanding of JumpStartoperation is required to understand thecustomizations made but the importantpoints are

Although typically associated withnetwork installs JumpStart is also apart of traditional installs of Solarisfrom CD-ROM installing from localmedia rather than the network

CART plugs into this JumpStartmechanism and replaces certainscripts so that JumpStart does notperform normal installation of pack-ages patches and so on Instead itprovides enough of a boot upprocess to get to the stage where aCART script can be run after whichJumpStart relinquishes control toCART

Future enhancements to CART includeintegration into a networked environ-ment and implementation on otherUNIX variants

A LINUX APPLIANCE CONSTRUCTION SET

Michael W Shaffer Agilent Labs RCS

The motivation for this project startedwith the authorrsquos need to support remoteinstallations of Linux servers providingfile print and network routing serviceslocated in areas with few skilled person-nel capable of disaster recovery

One way to address the support issue andestablish a fairly error-free disaster recov-ery process is to eliminate the traditionalinstall of the operating system and addi-tional software and instead boot and rundirectly from removable media using aLinux distribution configured for thispurpose

Rather than independently tuning aLinux distribution for each specific pur-pose ndash such as a print server ndash the scopeof the project was enlarged to create amore generic framework for creatingminimal Linux systems Hence the name

ldquoLinux Appliance Construction Kit(LxA)rdquo where the lsquoxrsquo represents the func-tion of the appliance (eg LPA == LinuxPrinting Appliance)

The design and implementation of LxAfollowed several principles

Systems are built by composition ofneeded pieces rather than reductionof a large set of components

Systems run from read-only andorremovable media (although harddrives may be used for swap vartmp and other transient storage)

Omit login and run-time configura-tion (except during developmentwhere facilities such as console loginand an interactive shell may beneeded for debugging)

Use modern standard componentssuch as the kernel libc and so on

A lot of work goes into determining whatis needed testing the images creatingbootable CDsfloppies and so on Theunderlying technique for running an LxAsystem is to use an initrd image and runthe entire system from it at boot time

There are numerous advantages to usingLxA over the more general purposeLinux installations and these were alsoimportant goals in the design

Reduced complexity which in turnenables better documentation and amore thorough understanding of thesystem

Reduced security vulnerabilityresulting from simplicity

Reduced setup maintenance andupgrade time

Reduced probability and impact ofhardware failures

Future work will address more types ofLxA appliances enhancements to theexisting LPA-CD appliance and auto-mated scripts for identifying componentsneeded for new LxA systems

More information about LxA can befound atlthttpwwwequusasinuscomlxagt

C

ON

FERE

NC

ERE

PORT

S


AUTOMATING DUAL BOOT (LINUX AND NT)INSTALLATIONS

Rajeev Agrawala Shaun Erickson and

Robert Fulmer LucentBell Labs

Research

Although tools are available to automatethe installation of Linux and NT thereare no good tools to automate the instal-lation of PCs that dual boot either Linuxor NT from separate partitions on a harddrive

This motivated support personnel atLucentBell Labs Research to design asolution to this problem as users werealready starting to use dual boot installa-tions of NT and Linux These wereinconsistently installed by differentadmins time-consuming to perform andnot reproducible

The solution employs automated installsstarting with a modified ldquobootnetrdquo RedHat Kickstart floppy The alternative is touse disk cloning but this requires similarhardware and peripherals cannot dealwith unique NT SIDs and involves a lotof work in updating an entire imagewhen any piece of software is changed

The process is designed to start with anadmin booting from a floppy and select-ing an install option (NT 4 Linux orboth) For dual boot installations thefirst OS installed is Linux using RedHatrsquos Kickstart Automated customiza-tion is performed the disk is reparti-tioned and a DOS file system is createdfor the second OS install namely NTNote that this file system is eventuallyconverted to NTFS

Some files required for the automatedNT installation are copied to this parti-tion and a reboot occurs to invoke theNT installer After the NT install and cus-tomization steps are complete a rebootsurrenders control to the first OS ndash Linuxndash where final configuration of X andaudio must be done manually due toproblems with lack of device driver sup-port that could interrupt the automatedinstallation

Some difficulties were encountered withpassing information about the installa-tion type to the Linux kernel creatingtwo primary partitions at once andlocating LILO in boot vs the MasterBoot Record The design accommodatessolutions andor workarounds for thesewhere necessary

The system has been in use for more thansix months and future plans include sup-port for automated X and audio configu-ration in Linux and the addition of otheroperating systems (eg Win 2000)

Source code and configuration profilesare available from the authors ltdualbootinforesearchbell-labscomgt

NETWORK TRACK

DEPLOYING QUALITY OF SERVICE FEATURES ON

YOUR NETWORK

Eliot Lear Cisco Systems

Summarized by Paul Federighi

Eliot Learrsquos talk described what quality ofservice (QoS) is why you might need itand the methods for achieving it Thetalk was mainly focused on QoS as it per-tains to voice communication on an IPnetwork though other types of data suchas video and Web traffic were also men-tioned

As Lear explained QoS is a method ofgiving preferential treatment to certaintypes of data on the network Applica-tions such as interactive voice and videohave special needs Voice has certainbandwidth and latency requirements andis drop sensitive Most packets mustmake it through with less than 200 ms oflatency This includes transmit time andany queuing delays QoS is not importantfor non-interactive traffic or non-time-critical traffic that can be buffered

Lear stressed the point that QoS featuresare needed on every piece of equipmentin the communications path where pack-ets can be queued This includes routersEthernet switches ATM switches framerelay etc

There are two models for achieving QoSintegrated services (Intserv) and differen-tiated services (Diffserv) With Intservthe application specifically requests (viaRSVP) resources at every hop along theway Call setup happens first thenreceivers request reservation (in bothdirections) Some of the advantages

Works with both unicast and multi-cast traffic

End points know early on whethertherersquos a reservation

By contrast Diffserv has no end-to-endsignaling Instead it uses per packetmarking rather than marking the entirestream Traffic is marked based on a pol-icy domain and is policed at the edgesSince there is no signaling an error couldcause an application to fail silently Oneneeds to pay careful attention to trafficengineering and either allow for addi-tional bandwidth on links or constraintraffic to predictable paths Packets areseparated into different classes

Best effort Assured forwarding (AF) ndash all data

will get there Expedited forwarding (EF) ndash

preferred over other traffic

Lear explained several different bufferingand queuing techniques on networkequipment The old way is to use a FIFOWhen congestion occurs the end of thetransmission gets dropped However thisis unfair to lower bandwidth protocolsand dropping packets just wastes band-width Methods for overcoming thisinclude random early detection (RED)weighted red priority queuing andweighted fair queuing Other methodswere mentioned as well

Management and monitoring are impor-tant with QoS You need to know if yourpackets are getting through and if thelatency is tolerable Tools are just startingto become available

Trying to achieve QoS across the Web hasscalability problems There is research

beginning with ldquobandwidth brokersrdquoRight now you canrsquot get QoS across mul-tiple providers Instead a good idea is todistribute data throughout the Web toreduce latency and get better bandwidthutilization

When asked about security Learresponded with the point that you canrsquotdo packet classification on encrypteddata

When asked about features to look for inhardware Lear mentioned several ques-tions to ask including ldquoAre there multi-ple output queuesrdquo and ldquoCan youclassify data in the output queuesrdquo Itrsquosalso important to remember that bottle-necks typically occur because of the linecards used not the backplane of thedevice

SESSION ANALYZE THIS

Summarized by Tony Katz

WIDE AREA NETWORK PACKET CAPTURE AND

ANALYSIS

Jon T Meek American Home Products

Corp

Jon Meekrsquos talk covered why we need toanalyze and how he went about doing itWe need to know what is happening onthe wire to diagnose such problems asslow applications and network conges-tion To monitor the frame networks Jonused a small PC running Redhat Linuxwhich was plugged into the CSUDSUvia a serial connection to capture HDLCpackets for analysis over time The topicof time interval was interesting You cancapture packets continuously whichtakes a lot of resources or in intervalssuch as 15 minutes or 10 seconds Whatinterval you choose can make a signifi-cant difference Jon did all of this using Cand Perl It is a fairly inexpensive way tomonitor your frame relay and get reason-ably good results


SEQUENCING OF CONFIGURATION OPERATIONS

FOR IP NETWORKS

P Krishnan IPSoft Inc T Naik Bell

Labs G Ramu CoSine Comm Inc

and R Sequeira IPSoft Inc

P Krishnan addresses the problem of los-ing segments when updating routingconfigurations across a complex networkif the updates do not happen fastenough The proposed solution issequencing This solution will work but itassumes many things eg that you areusing OSPF that routes are static routesetc This is achieved by indirect telnettraceroute and reverse traceroute Notbad if your environment fits all the criteria

ND A COMPREHENSIVE NETWORK ADMINIS-TRATION AND ANALYSIS TOOL

Ellen Mitchell Eric Nelson and David

Hess Texas AampM University

There are lots of vendors and even moresoftware out in the world and each onedoes something important but none ofthem do it all ND was designed by TexasAampM to accomplish this task Theywanted it to be powerful but low levelportable customizable and scalable Itwas written primarily in Python anduses SNMP SQL tables and a MySQLback-end database Python was used forits modularity With ND you can enableports configure new devices do script-ing but best of all is that it has built-indocumentation This is a great featurenot so much to point a finger at some-one but to know who to talk to aboutwhy a particular change was made TexasAampM is also looking to add event moni-toring to ND Currently ND is not avail-able but will probably be released to thepublic sometime in the future

SESSION GO WITH THE NETFLOW


COMBINING CISCO NETFLOW EXPORTS WITH

RELATIONAL DATABASE TECHNOLOGY FOR

USAGE STATISTICS INTRUSION DETECTIONAND NETWORK FORENSICS

Bill Nickless John-Paul Navarro and

Linda Winkler Argonne National

Laboratory

The first part of the presentation given byBill Nickless focused on the problem ofhaving a high performance network witha minimal firewall

Ciscorsquos NetFlow provides a summary ofdata traffic through a router This datamust be captured and analyzedArgonnersquos way to do this was through theuse of database technology Their hurdlesare the amount of data coming in andthe ability of the database to keep upThey went with a high-powered databaserunning on an SGI Origin 2000 Theyexperimented with both MySQL andOracle 8I but ended up using a SQLback-end database They used Perl scriptsto catch the data and feed it into thedatabase for analysis This worked verywell overall The big issue is that notevery site has an Origin to process thou-sands of records at a time Scalability isdetermined by your database applicationand tuning parameters

THE OSU FLOW-TOOLS PACKAGE AND

CISCO NETFLOW LOGS

Mark Fullmer OARnet and Steve

Romig Ohio State University

Steve Romig spoke about his applicationof NetFlow Their interest was more of asecurity focus They also had a volumi-nous influx of data but handled it a littledifferently OSU looked at aggregationcollection viewing and security using aset of tools they created called FlowTools They reduced the data load byaggregating the data into summariesThis allows viewing at any given pointTheir security features were the mostinteresting There has been a lot of focusput on incident response OSU used a

C

ON

FERE

NC

ERE

PORT

Svariety of Flow Tools to detect ldquointerest-ing network trafficrdquo host or IP range pro-filing as well as detecting networkattacks ie denial of service OSU is con-tinuing to expand their set of tools to domore in the realm of the clientserverrole on the end points of the wire For acloser look at the tool set check theirWeb site lthttpwwwnetohio-stateedusoftwaregt

FLOWSCAN A NETWORK TRAFFIC FLOW

REPORTING AND VISUALIZATION TOOL

Dave Plonka University of Wisconsin-

Madison

Dave Plonkarsquos presentation was savingthe best for last Not to say that the otherimplementations were bad but they didnot have a visualization component andDaversquos did The University of Wisconsin-Madison used an open systems softwarepackage called FlowScan FlowScan uses areport module called CampusIO to accu-mulate the raw flows and push the statis-tics to a round-robin database The datawas then taken by FlowScan andgraphed The graph which was colorcoded to traffic was able to show bothinbound and outbound traffic The col-ors of the graph differentiated betweenHTTP FTP and (the ever-popular) Nap-ster traffic The benefits of the graph overdata figures is that you can instantly seewhat type of traffic is using the mostbandwidth at any point in time Graph-ing also helps you pinpoint anomalieseasily Future expansion points forFlowScan are in the area of event notifi-cation or alerts For more information goto lthttpnetdoitwiscedu~plonkaFlowScangt

This was a great depiction of the uses ofCiscorsquos NetFlow It is a definite improve-ment over analysis by sniffer

BROADBAND CHANGES EVERYTHING


Brent Chapman Great Circle Associates

Brent Chapman spoke about how broad-band ndash which includes the variants ofDSL cable modems and possibly even


wireless ndash changes the way people per-ceive the Internet

Broadband has two features itrsquos highspeed and always on DSL providesspeeds on the order of 144Kbps (or morethan 7Mbps) Cable modems share thesame big pipe but provide similar highspeeds In comparison even the fastestphone-modems provide no more than53Kbps By ldquoalways onrdquo Brent means thattherersquos no longer any dial-up delay andno busy signals This makes the Internetlike electricity or water you flip a switchor turn a knob and itrsquos just there Thiswill change how people perceive and usethe Internet in the long run rather thansaying ldquoIrsquoll go online later and do thatrdquotheyrsquore much more likely to hop on andoff the Net for brief visits to accomplishtasks as they come up as opposed to wait-ing until later (Note that most consumerelectronics today ndash stereos televisionsand microwaves ndash donrsquot actually powerthemselves completely off They remainin a reduced-power ldquostand-byrdquo mode sothey can appear to power up morequickly when needed)

Broadband is also cheaper than tradi-tional leased lines A T1 line from atelecommunications provider (telco)used to run $1500 a month Comparablespeeds via DSL are on the order of $300 amonth

The revolution in providing broadbandleads to new capabilities such as con-necting small offices or home offices tothe Internet at high speeds as well asmaking telecommuting more effective forvirtually everyone It also leads to newservices or more efficient older servicessuch as

Streaming audio and video Backing up over the network (such

as Backup) Software auto-updates (Apple

Symantec) Push services (PointCast) Cooperative computing

(SETIhome) Interactive games

Unfortunately broadband also leads tonew security threats ldquoAlways onrdquo meansldquoalways vulnerablerdquo You can no longerassume that you can only be hit byattacks when yoursquore online in front of thecomputer when the Internet link isalways up Cable modem lines are sharedwithin a neighborhood so ldquoNetworkNeighborhoodrdquo takes on a whole newmeaning If you have shared your disk orprinter within your own home yoursquorealso sharing them with the entire cableneighborhood We should expect to seenew hardware and software firewalls builtinto broadband DSL in the near future

Broadband also allows you to savemoney Many homes have more than twocomputers so networking them withinthe home to share a single big pipe forbandwidth makes more sense to moreusers now This means that you couldcancel your second phone line (savingabout $15month) as well as multiple ISPaccounts (saving $20month)

Whatrsquos coming in the future of broad-band Brent expects that virtual ISPs (forsales and marketing features) affinityISPs (like credit cards) subsidizationand cross-marketing will happen in thenear term Wersquoll also see voice-over DSLand voice-over cable (some areas alreadyhave one or both of these) the problemfaced by the providers here is ldquofive 9s reli-abilityrdquo or less than five minutes ofdowntime ndash scheduled or unscheduled ndashper calendar year Wersquoll see more networkappliances (like WebTV and Tivo) andradio- and broadband-ready MP3receivers Wersquoll also see Internet-enabledappliances such as the refrigerator with atouch screen for restocking linked to agrocery delivery service such as Peapodor WebVan

There are several IT management issueswith broadband First among these issecurity should employeesrsquo homes beinside or outside the corporate firewallIf theyrsquore inside who other than theemployee has access to the company net-work If theyrsquore outside should the

corporate Internet access be shared withthe homes If so we need to have somekind of firewall protection (but then whomaintains and monitors those firewalls)if not the cost to the company will sky-rocket since every home user needs tohave their own bandwidth What carriersare available to the employee Who sup-ports and supplies the home systemHow can you provide mutually secureaccess such as when an employeersquosspouse works for the competition Is aVPN the right solution If so is it PC-based (which leads to driver issues) orrouter-based (which doesnrsquot address theother-people issue) Are personal fire-walls the answer Those also lead toissues of who provides configuresreconfigures manages and updatesthem and ignores the multiple-connec-tion issue

In the question and answer section Brentnoted that distributed denial of serviceattacks (DDoS) will increase Host-basedsecurity has to come back into style sincefirewalls are no longer enough protec-tion The CheswickBellovin model of acrunchy exterior and creamy interior nolonger applies Satellite broadband isunlikely because of the huge latencyinvolved Broadband affects the corerouters When asked what itrsquoll take toadminister the high-bandwidth providers(such as Akamai) Brent noted thattherersquos no good answer yet but we cer-tainly need to work on it As an exampleAkamai has 600 servers and is movingtoward 600000 servers Broadband alsoleads to more peer-to-peer networkingso the traditional source-and-sink modelmay need to be redefined


SECURITY TRACK

COPS ARE FROM MARS GURUS ARE FROM

PLUTO DEALING WITH ldquoTHE FEDSrdquo AND

OTHER COPS

Tom Perrine Pacific Institute of

Computer Security

Summarized by Dave Homoney

Tom made this a very interesting talk Hisinjection of real-world scenarios was veryhelpful The talk was geared towardsysadmins those of us who might have arun-in with a hacker and need to knowwhere to turn and the protocols to useHe also talked about what to do whenlaw enforcement (LE) comes to you

Tom described how to tell if a call fromthe FBI is a hoax if an FBI agent is call-ing you directly it probably is He statedthat most federal agencies will contactyou through a local law enforcementagency He also said that contact by theFBI would be through the nearest localoffice

Tom mentioned several cases in which LEscrewed up particularly the case againstSteve Jackson This case produced a lot ofnegative press for LE and marked thepoint at which LE moved from the gunsblazing approach to the computer savvyLE officer approach

Tom also talked about when not to helpLE stating that the first thing you need isa good lawyer You donrsquot want to do any-thing as ldquodirectedrdquo by LE or you could beconsidered an agent of the governmentcausing you problems in court Insteadyou should follow the directions of yourcompanyrsquos legal staff And of course youwill need to comply with all court orderssuch as subpoenas

Finally you should create a bridgebetween yourself and LE so that whenyou need them yoursquoll have a friendlycontact He mentioned several times thatldquothey are just like usrdquo adding that thereare always exceptions

DOES IT TAKE THE SAME SKILL SET TO SECURE

A SYSTEM AS TO BREAK INTO IT

Panelists Peter Shipley Lab OneSecureInc Mark Hardy Guardent Inc andElias Levy securityfocus

Summarized by Dave McFerren

Some of the topics the panel discussedwere

Should companies hire crackers tocatch other crackers

Can you trust someone who wasonce a cracker

Discussion was fairly one-sided concern-ing the skill set needed to break in com-pared to the skill set to secure Theoverwhelming majority of opinion wasthat cracking requires only a subset ofthe skills needed to secure systems thereare many different ways to compromise acomputer and a cracker needs only con-centrate on one particular service thatthe computer may deliver Another topictossed about was the question of whetheryou can hire ldquoblack hatsrdquo to do ldquowhitehatrdquo jobs Although there are manystartup or fringe companies that tend todo this the general consensus was thatyou should become a white hat by your-self and make a foray into the corporateworld before earning the trust of theldquosuitsrdquo

The most interesting discussion arose inresponse to the question ldquoWhat does ittake to become a security expertrdquo Onepanelist ndash who had previously been ablack hat ndash insisted that you had tobreathe live and eat security for years tobecome good at it Others disagreedbelieving that you can have a life withfamily and friends and still be able to doa good job at security But most agreedthat since security requires more than thetraditional 40 hrswk a person wouldhave to be at least somewhat obsessedwith the subject to be really successful

Overall the discussion was interestingalthough I was not sure I got more thanthe single perspective held by the com-puter security profession But it did show

C

ON

FERE

NC

ERE

PORT

Sme the ldquoother siderdquo of the security issuesthat I deal with on a day-to-day basis

REAL-WORLD INTRUSION DETECTION ndash FIRST STEPS

Mark K Mellis SystemExperts

Corporation

Summarized by Steve Wormley

Mark Mellis covered much of what isneeded to set up intrusion detectionusing primarily free products on a small-to medium-sized network He noted thisdiscussion didnrsquot apply to larger sitesbecause they generally have larger prob-lems but the basics are the same

He first gave an overview of why onewould do intrusion detection (ID) Basi-cally the crackers are becoming moresophisticated the networks are becomingmore complex more protocols are flow-ing through the firewall and there aremore connections to business partnersand other points of attack

The assumptions for this talk were thatthe solution needed to be cheap that theSA was familiar with ftp and make andnormal freewareopen source setup andthat the admin was busy and ID was apart-time job

ID systems should tell you when realthreats occur but be able to log even doorrattling Important things to trigger onare config changes auth failures attemptsto probe the site and attempts to accessservices

The things he recommended deployingincluded centralized syslog functions(UDP syslog or nsyslogd) a log-analysisproduct like log_analysis or log surfer atripwire like Tripwire or aide Klaxontomonitor for connection attempts onunused ports sscanlogd to monitor forportscans and a product like snortwhich is a lightweight network IDS

A couple of points to remember routersare hosts too and need to be monitoredand can use syslog to do so To captureauthentication failures brute force does


work Donrsquot forget application exploitsscan the Web and appserver logs for any-thing out of place

Finally all exposed and all infrastructuremachines should be running host-basedID and network ID systems should beput where traffic is both concentratedand sensitive And of course anythingthat can be done is better than nothing

This was a good overview of the productsavailable and things that can be used inID It was a good talk for anyone whoneeds to install this type of service on asmall scale or as a precursor to learninghow to do it in a large environment

Mark K Mellisrsquo URL is lthttpwwwsystemexpertscomgt

SESSION SOMEONErsquoS KNOCKING AT

THE DOOR

Summarized by Eric Lakin

TRACING ANONYMOUS PACKETS TO THEIR

APPROXIMATE SOURCE

Hal Burch Carnegie Mellon University


This paper was one of two papers toreceive the ldquoBest Paperrdquo award for LISA2000 It was based on work done a coupleof years ago within Lucentrsquos corporatenetwork concerned with ways to find thesource of a denial of service (DoS) attackwhen the source of the packets is forged(the norm) Some of the assumptionsmade in the research which limit the use-fulness of the technique against distrib-uted denial of service (DDoS) attacksseen recently include the following thesource of the DoS packets is a singlesource no modifications to the currentnetwork infrastructure can be made(router or protocol) the attack is longterm and the packet-rate is constantFurther only DoS attacks that seek tooverwhelm the victim with bogus packetsare considered attacks that attempt tocause a malfunction by specially craftedinput are not

When considering the network topologyfor the purposes of a DoS attack it wasconvenient for the authors to describe itin terms of a tree graph The victimrsquosmachine is at the root of the tree withnetwork nodes being nodes in the treeEach path out of the victims local net-work subnet is a branch in the tree withfurther branchings being paths out of theconnected subnets and so on One pathto a leaf node is the attacking host

Because the source of the DoS packets isalmost always spoofed the assistance ofthe ISPs and network administrators out-side the victimrsquos network are usuallyrequired to locate and shut down theattacker There is often little motivationfor these people to help and even findingthe appropriate person to help may be achallenge If the appropriate people arefound and are willing to help they caneither put their routers into debug modeto determine which path the attack istaking or selectively cut off paths brieflyand see if the attack slows or stops

When the outside network managerscannot be contacted for some reason is itpossible to determine the approximatelocation of the attacker without physicalaccess to the outside network or theirrouters The authors of the paper wereable to come up with a way to selectivelyldquodeactivaterdquo a line remotely by using theldquochargenrdquo service and UDP broadcastpackets to selectively overwhelm or DoSa branch in the tree By selectively over-loading individual branches of the treeand watching the rate of incoming pack-ets one can determine with increasingaccuracy where the attacker is located

Because of the method used to over-whelm the branches accuracy is limitedto determining the subnet of the attackerThis however is enough to allow con-tacting of the appropriate ISP In simu-lated DoS attacks within Lucentrsquosnetwork the authors were able to findthe attackerrsquos subnet three out of fivetimes and were always able to determinethe subnet within two to three hops

The ethicality of this procedure wasbriefly touched upon and it wasacknowledged that this was of more aca-demic interest ndash and should never beused in real situations on the InternetFurther the program written to do thetesting and analysis is not going to bereleased

ANALYZING DISTRIBUTED DENIAL OF SERVICE

TOOLS THE SHAFT CASE

Sven Dietrich NASA Goddard Space

Flight Center

The purpose of a denial of service (DoS)attack is to overwhelm the victim so thatit is unresponsive to legitimate users orto construct input to make the victimhost act strangely or unreliably The for-mer is more common general purposeand what this paper focuses on

The simplest form of DoS involves anattacking host sending packets directly toa victim host As it was not always possi-ble for a single host to saturate a victimfurther refinements were made to DoSmethods Using amplifiers ndash hosts thatamplify the amount of traffic output byan attacking host ndash became commonwith the ldquosmurfrdquo attack Another possiblemethod was to coordinate among multi-ple attackers to concentrate on a specificvictim However in the past such coordi-nation has been largely manual such asagreeing to a time and victim throughIRC

Distributed DoS attacks are a recentldquoinnovationrdquo in the DoS toolkit In theDDoS model one or more attackers relaycommands to a ldquohandlerrdquo host whichmaintains a list of ldquoagentsrdquo ndash compro-mised machines running software toattack victim hosts Through DDoS soft-ware one individual can direct tens orhundreds of machines to attack targetswith the multiple sources of the attackmaking it highly effective and extremelydifficult to stop

The DDoS tool analyzed in-depth by thepaper was called ldquoshaftrdquo and was the sec-ond such software available after the


original trinoo Most analysis of shaftwatched traffic between a compromisedagent and a handler as well as actualattacks which the agent participated inAnalysis revealed the attack methods theshaft tool used ndash a combination of TCPUDP and ICMP flooding ndash and the com-munication channel between the agentand handler was discovered

SESSION DONrsquoT LET THEM IN

Summarized by John Ouellette

YASSP A TOOL FOR IMPROVING SOLARIS

SECURITY

Jean Chouanard Xerox PARC

Purpose to correct Solaris defaults

Permissive file modes Too many services running Inconsistent logging

Philosophy

One config file Model after Sun package ndash easy to

uninstall Tolerant about what it expects Server or workstation based

YASSPrsquos project

SECLean ndash core package Other RCS openssh Tripwire

tcpd+rpcbind

Goal to control startup of init scriptswhile allowing the machine to return toits pre-YASSP state

Needs more English-friendly docs andtesting for non-default Solaris systems

Additional information on YASSP can befound at lthttpyasspparcxeroxcomgt

NOOSE ndash NETWORK OBJECT-ORIENTED

SECURITY EXAMINER

Bruce Barnett General Electric Corpo-

rate Research amp Development

Purpose to present tools that are lacking

Content and function distributed coop-erative engine Dispatcher IW GUIThere are presently 21000 lines ofPerlTk code at ldquoresearchrdquo quality level

This tool checks for patches generatedfrom the Sun FTP site looks for Trojanhorses parses start files tracks variables$PATH etc examines rhosts under-stands NIS netgroups checks NFS accessto usersrsquo homes

Problems single-threaded not secure

Performance host with 2000 accountstook 30 minutes to check 5000 vulnera-bilities

Future will be TCP based multi-threaded

Conclusion object orientation is key toreusable algorithms

SUBDOMAIN PARSIMONIOUS SERVER

SECURITY

Crispin Cowan Steve Beattie Greg

Kroah-Hartman Calton Pu Perry

Wagle and Virgil Gligor WireX

Communications Inc

Problem granting least privilege notalways easy or feasiblepossible Forexample mod_perl runs at Apache levelof permissions

Solution ACLs for programs instead ofusers Subdomain is a kernel-levelenhancement to confine programs

For instance program foo can berestricted by a config file

foo etcreadme retcwriteme w

This gives program foo read access toetcreadme and write access toetcwriteme by invoking the chhat()method (ie change hat)

WORKSHOP SERIES

WORKSHOP 2 TEACHING SYSTEM ADMINISTRATION

Coordinators Curt Freeland University

of Notre Dame and John Sechrest

PEAK Inc


This workshop was a continuation of lastyearrsquos work The goal this time was tobrainstorm ideas for class material exam

C

ON

FERE

NC

ERE

PORT

Squestions concepts etc There was agood representation from all aspects ofsystem administration education with aheavy concentration of collegeuniversityeducators Here is the outline of theworkshop

Session 1Concepts and prerequisites

Session 2 Concepts taught War stories Best exam question What I wish class looked like Getting students to participate in class Measures of success Questions for a prerequisite (minimumcompetency) exam

Session 3 Tools we could use Develop examples

Session 4 Develop examhomeworkproject What we would like to have for a pro-gramtrackseriesconcentration Workshop review

Results of this workshop will be pub-lished under the mailing list of the work-ing group(ltsysadm-educationmaillistpeakorggt)and will be part along with the SysadmTaxonomy Project Sysadm CertificationProject and Benchmarking and Mea-surement Project of SAGErsquos efforts toformalize the system administration pro-fession

WORKSHOP 3 METALISA

Coordinators Cat Okita Global Cross-ing and Tom Limoncelli LumetaLucentTechnologies


Six major topics were covered

Staying Technical

We had a good hour-plus discussion onhow to stay technical and manage yourboss Some subjects that came upincluded having both responsibility andauthority getting someone to do the


nontechnical aspects while you concen-trate on the technical ones defining andreviewing roles and responsibilities usingagendas to run and control meetingsknowing when to say ldquoI donrsquot knowrdquoholding regular ldquotown hallrdquo type meet-ings for the user community and apolo-gizing when you screw something up

Retention Hiring and ldquoCoaching Outrdquo

The general consensus here was toinvolve the Legal and Human Resourcesdepartments as soon as possible and todocument everything If you need toencourage someone to leave you caneither treat it as a pure performance issueor possibly a security issue (for exampleif the employee in question has root) Ifperformance is an issue you can useimprovement plans or a probationaryperiod Yoursquoll definitely have to managethe morale of those who stay

This led to a discussion on hiring Firsthow do you find qualified applicantsYou can look online though for moresignal and less noise you can go by wordof mouth the SAGE jobs list and evencampus recruiters Second how do youconvince these qualified folks to joinyour company Some thoughts for man-agers focused on looking at the longterm not the short term since you can-not easily get rid of someone yoursquovehired do you have to raise or adjust thesalary of the new hire of the rest of theteam do you have to train people Arebonuses involved

This led to a discussion on retention Inorder to keep employees on your teamwe determined that managers need tohave flexibility in providing raises (bothin terms of frequency and amount) andreviews (more often than just annually)Providing perks such as training confer-ences laptops high-speed networkaccess soda and beer flex time toys toplay with (both computer-related andnon-) cool projects good managementgood co-workers good environment and

respect and recognition can help youretain your best employees

Leaving Gracefully

If you find yourself in the position ofleaving a job you should leave gracefullyHand off your responsibilities make sureno batch or cron jobs run from your ownaccount document everything (bothwhat happens and why) and put read-mefiles in nonstandard directories and hostsBe professional and do whatrsquos right forthe company donrsquot send any hate mailWhenever possible you should train yourreplacement As a manager you need toplan for your people leaving be it by leav-ing the department leaving the companyor being promoted out of a positionSome other questions that aroseincluded

How do you decide when to quit How do you handle a subordinate

being promoted over you How do you manage friends

Talking (Up) to Management

When talking to your management youhave to remember to tune to the audi-ence Talk about the technical issue interms that your audience is familiar withYou need to focus on the business rea-sons and issues and not the technical jar-gon You should ask your peers or evenyour boss to review any messages yoursquoreabout to send Find something in com-mon with the manager and use that as abasis for establishing rapport in yourcommunications

Talking to Now-Subordinate Peers

When yoursquore the one whorsquos been pro-moted to lead your team there are a fewthings you need to remember You haveto be careful with social events as a man-ager yoursquore no longer ldquoone of the gangrdquoThere is going to be some informationyou cannot share with your team andthey are going to know that You have totreat everyone the same regardless of howyou may feel toward them (such asfriends and non-friends in the sameteam) The dynamics will differ by group

size managing one or two people is dif-ferent from managing 10 or 12 Finallyyou have to be objective and impartial

General Tips and Techniques

We wrapped up the day with some gen-eral tips and techniques for being goodmanagers

Set boundaries for your team Let your employees fail Remember to test and include test-

ing in projects Consider when to delegate or take

over something Review your team your peers and

your management Spend a lot of time up front on con-

cepts and requirements of projects Protect or insulate or buffer the team

from more-senior management Back up (support) your employees

trust your team Stay out of the office to (1) delegate

and (2) find out what does anddoesnrsquot work without you

Donrsquot micro-manage your employees

Communicate up down and acrossopen communications are veryimportant

Teach skills not things or details Donrsquot lie to yourself Assume the other party is trying to

do whatrsquos right for the company Bounce thoughts off peer-level

managers Find a mentor (either inside or out-

side your organization) Do one thing every day that scares

you Match customer expectations with

reality prioritize Rotate your team through the vari-

ous positions to reduce the risk ofburnout

Donrsquot decree decisions unless youhave no time to reach consensus youcannot reach a consensus or there isan obvious violation ofpolicy


Kill off or postpone projects whennecessary

WORKSHOP 5 ADVANCED TOPICS

Coordinator Adam Moskowitz LION

Bioscience Research Inc


The professionalization of systemsadministration was one of the topics dis-cussed A comparison was made to doc-tors We use similar skill sets ndash diagnosiscomparability problem solving and soon But can it be said that lives are atstake when systems administrators dotheir job Doctors charge by the visit orthe procedure systems administratorsdonrsquot The models are however converg-ing in some ways Many systems adminis-trators are more concerned witharchitecture than are doctors There aredifferences in scale doctors are like helpdesks while systems administrators tendto serve larger numbers of people Doc-tors are in fact certified Some systemsadministrators contravene organizationalpolicies Doctors are liable lawyers areliable and engineers are liable systemsadministrators are rarely liable This ledto a discussion on professions profes-sions have standards for training andknowledge (certification) therersquos a fixedset of information Sysadmins are oftengrassroots with self-training and appren-ticeship Certification is a required stepping stone Maybe systems adminis-tration should be a ldquoguildrdquo Or maybe weshould form a union

A second area of discussion was whetheror not ISPs are now perceived as com-modities and whether they can be run ascommodities The consensus was thatthey can but you should be sure to checkout their long-term business prospectsbecause business models change rapidlyFinding a provider for services ldquobeyondthe basicsrdquo is hard ISP consolidation is inprogress Any new ISP will require newtechnology Not only are ISPs perceivedto be commodities so are their users(who are traded) Local and national ISPs

can survive but itrsquos tough for regionalISPs who are neither local nor a brandname Are there brokers for customersThere are special deals among ISPs butno B2B site DSL was enabled by aggre-gating terminations at the central officeThose who can scale will survive You cannow purchase a turn-key 10- to 50K-userISP solution that requires very low levelsof sysadmin skill Shell accounts are athing of the past people are runningtheir own servers at the end of a DSLline

A third major area of discussion was onseparating policy and implementationOne possible solution is to have an inter-preted ldquopolicy languagerdquo Maybe you canuse general principles and then color thebottom-level implementation to matchexisting policies This is more of a mind-set problem than a coding problem Letrsquosbuild policy engines not engineeraccounting (or whatever) systemsCfengine has features that can help youimplement policies You must codify thepolicy in a way thatrsquos measurable so youknow if yoursquore ldquoon policyrdquo or not (andthen you can get back on policy if you getfar enough ldquooff trackrdquo) Wersquore alreadyadapting host-based tools that querydirectories Maybe we can graft policyengines onto directory responses

A fourth area of discussion was on hownew technologies in the last few yearsseem to be languages This is truebecause languages can express extensibleideas they build from primitives andmove to greater complexity Some peoplesay ldquouse a database for policyrdquo but data-bases too often require predefinitionsLanguages on the other hand are infi-nitely extensible We think this is thesolution for policy expression A well-crafted language could potentiallyaddress this problem but we donrsquot knowof one right now We think languages canexpress these specifications at the properlevel The real problem is the ability todescribe when a particular operation isauthorized We need to agitate for rich-

C

ON

FERE

NC

ERE

PORT

Sness of expression in commercial toolsWindows has a lot of configurableoptions under the hood that were diffi-cult to access via the desktop or com-mand line even though an API wasavailable Declarative languages like Pro-log might be able to help here Excep-tions are surely the difficult andimportant part of this problem

We wrapped up by looking at our 1998and 1999 predictions to see if we werelate or still wrong We still have moremisses than hits In 1999 9 of our 19 pre-dictions came true (or mostly true) for a47 success rate More of our 1998 pre-dictions came true in 2000 but wersquore stilllooking at about a 50 success rate

Our predictions for 2001 are

Peer to peer (including systemsadministration) will grow thenshrink (85)

DSL-based ISPs will grow in popu-larity then die as the tech-savvy turntheir DSL to a friends-and-familyISP (65)

Alternate dialtone-to-home compe-tition will break loose (50 of themarket) this year ndash telephone com-panies will have to change their busi-ness plans (100)

The number of purported PDA- andmanaged home-appliance systemswill double in the next 12 months(75)

Some time this Christmas seasonthings will go well with e-commerce(100)

Therersquoll be an e-commerce disastersome time in the next year thoughOr a Fortune 100 company will havean above-the-fold e-disaster (40think itrsquoll show up in print)

We will have at least one Microsoft-facilitated security bug agrave la Melissaor ILOVEYOU in the next year(100)

Therersquoll be at least one major SiliconValley power outage that provides


above-the-fold problems for at leastone company (85)

Many dot-coms with otherwise prof-itable viable business models willfail because their names arenrsquot AOLor Yahoo (investor confidence willdrop further) (70)

This is NOT the year that SiliconValley loses its shine and people starta mass exodus (75)

80211b will become standard on allbusiness desktops and laptops in thenext year (80 on desktops 100on laptops)

80211b public Internet access willbe available in the top 25 US airportsby December 2001 (100)

A huge mobile phone will be dug upon the surface of the moon

You will not see networks on air-planes in 2001 (not counting dial-out via airphone) (100)

Businesses will find their storage (atleast) doubling this year with theconcomitant backup problems(75)

Linux will splinter (speciate) (10) The price of 17-inch flat panels will

come down to $700 (100) 200-dpi-resolution displays will

appear on desktops (10) Gigabit Ethernet hubs will dramati-

cally decline in price in 2001 (35) Serialization of Object Application

Protocol (SOAP) ndash RPC over HTTPwill ascend to wide acceptance(15)

Official or commercial music deliv-ery services will fail (85)

Finally we listed some cool tools wersquoreusing

VMware Rethinking the world in terms of

PHP MySQL and HTTP Wireless everything (80211b) Some of the new load-management

tools (batch queuing tools) for clusters

65-pound brute-breaker demolitionhammer with its own cart

PLSQL XML and JavaScript Wiki (a very simple browser-based

editing environment) VNC (virtual network consoles for

NT) Baytech power strips with Ethernet

access to remotely reboot via poweroutlet (vector for the Microsoft secu-rity outage wersquore predicting)

Blackberry (two-way) pager Unison (file synchronizer tool

between desktop and laptop two-way comparison etc)

Python PalmHandspring ssh in IOS Netflow (Cisco-created protocol

gaining acceptance) tools thatrsquovecome out in the past year you cannow do accounting without trashingperformance

tangram(lthttpwwwtangramorggt) a Perlmodule that makes variables to per-sistent storage (SQL database)

Herman-Miller Aeron chair itrsquosreally worth it if you sit on your assall day

Authenticated Web environmentthat keeps the authentication tokenon all the time

Win32rsquos NetCaptor tabbed Web-browsing Web interface

Win32rsquos PowerMarks bookmarkmanager that has keyword-basedsearches

blog (short for weblog) that logswhere you go and lets you store abunch of stuff in a column as if youwere writing a letter or column likeslashdot similar to userland (seelthttpwwwuserlandcomgt fordetails) Can be published as RSSfeeds to JavaScript news-ticker appli-cations

Newsbytes-style column

WORKSHOP 8 SYSTEM ADMINISTRATION

PROCESS IMPROVEMENT BENCHMARKINGAND METRICS

Coordinators Carolyn Hennings

Megapipe Tom Limoncelli Lumeta

Lucent Technologies and Alva L Couch

Tufts University

Summarized by Nicholas Schrieber

Caveat this report is not intended to beminutes of the meeting

Some workshops are odd especially thefirst one on a topic with no clear agendaor mission The mission options at thestart range from disbanding at the end ofthe day saying ldquoOK wersquove exhaustedthat topicrdquo all the way to planting theseed of what eventually becomes a 500-member consortium with an executivedirector

The eight of us who attended spent muchof the day exploring an appropriate mis-sion for the group We examined andcompared possible forms that a useful SAprocess improvement program or docu-ment could take and we tried to deter-mine appropriate roles for the committeewithin SAGE and vice-versa The initialmission statement we decided on for ourdayrsquos work was ldquoDevelop a frameworkand methodology for measuring andimproving organizational maturity withrespect to SA practicesrdquo At the end of theday it was very clear that we had notactually even begun a framework andmethodology but we had reached somedecisions on the shape it should take aswell as the role it should play vis-agrave-visSAGE and several other SAGE-recog-nized programs

Among the various models we had toconsider were the System AdministrationMaturity Model (1993) by CarolKubicki the SEI Software CapabilityMaturity Model upon which it wasbased some high-level process measure-ment techniques a committee memberpresented as well as the Systems Admin-istration Body of Knowledge for whichGeoff Halprin is preparing a Guide docu-


ment We also needed to consider theSAGE taxonomy project and appropriatecomplementarity with it

We need to be able to ensure repro-ducible results not just repeatability ofprocess This distinction represents aneasy shortcoming many such modelscould harbor Those of us who had readCarolrsquos 1993 SAMM were in consensusthat it was a good solid attempt at whatwas needed There were two problems wesaw one being the state of the systemsadministration industry at that time andthe other is that even now the model asshe presented it is probably not clearenough to make its widespread adoptionlikely We seemed to agree that the stateof the industry has advanced and eventhe codification of the in-progress SABOK suggests wersquore now ready to applyCarolrsquos SAMM principles But they needto be clearer less academic and probablyshould incorporate some changes thatadditional years and a larger committeecan bring to the project

The final outcome of the day was multi-part

We would work toward the develop-ment of a new SAMM documentthat is essentially reflective of the1993 document but greatlyimproved

We would ask SAGE to officiallysponsor the project which would initself facilitate the status of the docu-ment toward becoming a standardWe consider that this sponsorship isalready basically a fact but requiresclarification Perhaps the best term isldquoSAGE-recognizedrdquo rather thansponsored

Further we would ask SAGE forfunding that might be necessary toprovide staffing for further develop-ment of the new SAMM document

Carolyn Hennings would write a for-mal proposal probably for presenta-tion at the February SAGE boardmeeting

Other loose notes from the day include

Part of this job is the creation of a met-rics language This obviously dovetailsvery significantly with the taxonomyproject We need to take a step back andinternalize existing taxonomy

Alva Couch presented some interestingideas on metric (measurement) systemsHe pointed out that the biggest problemis too many variables

As a high-level summary he made refer-ence to the Boehm approach from SEand proposed a modal approach A majoraim would be to relate life cycle cost overideal cost (complexity of task) His con-clusions

Robustness ability to survivechanges = cost of replacementcom-plexity of assigned tasks

Efficiency relative cost of life cycleto ideal

Over the long run the group would haveto deal with complex couplings of majorissues For example coupling with the SABOK project is on the one hand obviousand perhaps inevitable But it could raisefalse presumptions about their compati-bility or the ldquoownershiprdquo of the SA guideproject Geoff Halprin is currently theauthor and owner although he expects toallow free use of it as long as his author-ship is respected In the Project Manage-ment field the PMI owns the guide to thePM BOK and as such it does not act as aprivate person or corporation but moreas a standards group

WORKSHOP 9 TAXONOMY

Coordinator Rob Kolstad

Summarized by Dave Bianchi

The goal of the workshop was to create aframework for a taxonomy of systemsmanagement to enumerate all the tasksthat a systems manager might perform

The eventual goal of the taxonomy proj-ect is to provide a list of tasks that a sys-tem manager performs along with ashort description of each task Rob would

C

ON

FERE

NC

ERE

PORT

Slike to have a fairly complete list of taskswithin three months

The first half of the day was a brain-storming session naming tasks and try-ing to group them in some meaningfulway The group came up with a grid sys-tem to categorize common activitiesaround a task Common activitiesincluded policy standards securitybudgeting and legal issues

The second half of the day was spentfocusing on just one task ldquoPrintersrdquo Robproposed that a Web site be created toallow a group of people to work on therest of the tasks and he volunteered tocreate the Web pages The group volun-teered to help maintain the Web pagesover the next three months

See lthttpacedeloscomtaxongategt

BOFS

FREEBSD


The FreeBSD BoF was chaired by twoFreeBSD developers one of whom is acore member (David Greenman) Ques-tions were mostly directed at the twodevelopers with only minor help fromthe audience For some questions thiswas appropriate ndash such as the currentldquostaterdquo of FreeBSD and Core and theprogress of BSDi integration intoFreeBSD

ldquoCorerdquo is a group of developers that over-sees the direction of FreeBSD develop-ment Until recently a person joined thecore by developing and proving theirability and then being asked to join Inthe past year there was a first-ever elec-tion of the core (by the active developersI believe)

Expectations for the integration ofBSDiFreeBSD following the merger ofWalnut Creek CDROM and BSDi havefailed to materialize A rewrite of theSMP sections of the kernel are currentlyin progress with design influences from


the BSDOS code but no actual codemingling has occurred yet

Of particular interest was the discussionof a logging or journaled file system inFreeBSD on-hand in the audience wasKirk McKusick the architect of the BSDFast File System He gave an overview ofSoftUpdates an FFS addition currentlyavailable in FreeBSD that increasesperformance and addresses some of theissues that relate specifically to journalingfile systems Theoretically at least fsckinga file system shouldnrsquot be necessary whensoftupdates are enabled but fscking iscurrently still done ldquojust in caserdquo until thecode and theory have proven themselves

The remaining discussions mostly relatedto individual problems with FreeBSDmost of which boiled down to ldquomoreinformation neededrdquo

GETTING PLUGGED INTO SENDMAIL

Mike Smith ActiveState


This BoF was actually titled ldquolibmilterGetting Plugged Into Sendmailrdquo Libmil-ter is included in Sendmail 810 andabove and gives access to the SendmailMail Filter (milter) API Milter giveshooks into every stage of an SMTP trans-action and lets you perform customactions based on any part of said transac-tion including the content of the mes-sage While some of this is possiblewithout milter (for example procmaillets you filter after message acceptance)milter gives you more functionalitybefore the message has been accepted bythe mail server

Some things you can do with milter

Mail archiving selectively archivemessages based on specific criteria

Spam control deny mail deliverybased on your programmed specifi-cations

Content rewriting add new headersor change existing content

Virus scanning verify that incom-ingoutgoing attachments are clean

All of the examples shown during theBoF were written using ActiveStatersquosPerlMX product which allows these fil-ters to be written in Perl instead of theusual C It is a commercial product but isavailable for free to individuals and edu-cational sites For more information seelthttpwwwactivestatecomPerlMXgtand libmilterREADME from the Send-mail distribution

BOF NETBACKUP

Curtis Preston Collective Technologies


Due to some scheduling confusionPreston was late coming to this BoF butin normal USENIX fashion the attendees(the room was very full) proceeded torun the BoF themselves and a lively dis-cussion of various NetBackup technicaland support issues began This mostlyrevolved around the typical vendor com-plaints and a few specific issues withbackup scheduling and system security inNetBackup

After Curtis arrived the discussionquickly became more of a tutorial whichwas quite interesting Most of the infor-mation was on bpgp an undocumentedcommand in NetBackup which is usedinternally to copy files between systemsAlthough one could consider bpgp to bea security hole (at least on NetBackupsystems which are not using the built-inauthenticated communication methods)it is a very useful tool for copying config-uration files and other informationbetween systems which are NetBackupclients or servers Some typical usesinclude dissemination of exclude filesand any other sort of configurationinformation In all this was a good andworthwhile session though it could havebeen much more effective if it had beenscheduled for longer than an hour

POSTMASTER

Strata Rose Chalup VirtualNet

Consulting


This BoF was intended to get peopletogether to talk about SAGErsquos upcomingldquoRole of Postmasterrdquo booklet The book-let is meant as a ldquobest-practicesrdquo guide forsystem administrators in charge of mailsystems but will not be limited to anyparticular mail transport agent (MTA)The booklet is being co-authored byStrata Rose Chalup and Brian KirouacThere were far too many suggestions andplanned topics to be listed here Thisbooklet might actually have to be splitinto multiple booklets to cover all of thedifferent parts of the postmaster role Ifyoursquore interested in becoming involvedwith the booklet (either by making sub-missionssuggestions or just seeing whateveryone else is sending in) please con-tact Strata via emailltstratavirtualnetgt

SENDMAIL ndash OPEN SOURCE COMMERCIAL

Summarized by Brian Kirouac

Open Source 812

new IO library new memory management to limit

forks no longer need to get sfio for secu-

rity add-ons

Open Source 9x

global optimization some things from qmail and postfix not a monolithic program but will

not go as far as postfix (ldquopostfix hastoo many small processesrdquo)

most processes will be threaded some platforms will be dropped new configuration files ambitious goal for performance

machines two or three generationsout will be able to handle 100 mil-lion messages a day and this willwork on clusters


SMI Advance Messaging Server

release 25weeks ago

SMI

Eric is being cautious he hasnrsquot killed the new CEO yet still in the process of going through

ripples of new CEO ldquostill going wonderfullyrdquo

The last CERT advisory for Sendmail wasin January of 1997

LISA 2000 SOLARIS DOCS


The Solaris Docs BoF was held by severalof the Sun documentation developersThe primary purpose of the BoF was tosolicit input from Solaris Documentationusers and to answer questions theyposed

Several issues were raised during the BoFincluding

Jumpstart documentation is out ofdate Attempts are being made toimprove it for Solaris 9

There were several complaints aboutthe navigation on suncom in gen-eral and on docssuncom in partic-ular

Several users wanted documentationfor End-of-Lifersquod products keptonline so that users of older hard-ware can still obtain information ontheir products

One of the primary improvementsmade in the last several documenta-tion releases was to try to includemore task-specific information Thiswas prompted by user requests

There should be several upcomingimprovements in the BigAdmin site atSun including tying BigAdmin into theother documentation sites and providingmore information on upcoming Solarisreleases BigAdmin is a useful Solarisadministration resource and is located atlthttpwwwsuncombigadmingt

Email comments on documentation canbe sent via the comment alias ondocssuncom or by sending directly tothe documentation developers

ltcindyswearingensuncomgtltjulienelsonsuncomgtltkathyslatterysuncomgt

UNIX ON HANDHELDS


Steve Wormley led a BoF on ldquoUNIX onHandhelds and Wearablesrdquo Wednesdayevening It turned out to be mostly onwearables by a guy from MIT (next timejust handhelds will be covered)

It was an interesting group with lots ofquestions and details of what is beingdone in the field The head-mounted dis-plays are getting smaller the networking(80211 and CDPD) is getting morewidespread and the devices are gettingmore powerful

All in all it was interesting but a bit shorton handhelds and they definitely stillarenrsquot for everyone and not even formost of us yet

4 Vol 26 No 2 login

OUR THANKS TO THE SUMMARIZERS

Lee AmatangeloBrian BaggettDave BianchiPaul Federighi Jim FlanaganSteve HansonDave HomoneyTony KatzBrendan KelliherBrian KirouacVinod KuttyEric LakinDave McFerrenJohn OuelletteSocrates PichardoNicholas SchrieberSam ShafferJosh SimonDoug StewartTheo van DinterCraig VershonSteve Wormley

Photographs taken at LISA 2000 can be

found at

lthttpwwwusenixorgpublicationslibraryproceedingslisa2000confpixpixindexhtmlgt

conference reportsLumeta Corporation Both papers had astudent co-author

Dan Geer USENIX president announcedthe proposed split of USENIX and SAGEcovered elsewhere in this issue BarbDijker SAGE president echoed Danrsquosannouncement reiterated the desire formember feedback and announced the2000 SAGE Award for outstandingachievement Celeste Stokely for herwork in collecting and distributing sys-tems administration information for over10 years

KEYNOTE ADDRESS

THE WORLDWIDE SYNDICATE

JD ldquoIlliadrdquo Frazer User Friendly


The artist behind the immensely popularWeb cartoon strip ldquoUser Friendlyrdquo(lthttpwwwuserfriendlycomgt) relatedhow he ventured into self-syndicating hiswork and what he has learned about thetraditional syndication model along theway ldquoUser Friendlyrdquo recently celebratedits third anniversary and has experiencedexplosive growth in Web page impres-sions

Cartoonists are curious about humannature and the traditional syndicationprocess causes these people no end ofpain by imposing on them a fundamentalseparation from their audience which istreated as a market rather than a com-munity Each layer between cartoonistand audience removes revenue alongwith artistic control over content JDtold how Berke Brethed stopped doinghis popular comic ldquoBloom Countyrdquo notbecause he ran out of material or burnedout but because some of the contentoffended Donald Trump who in turnbought the syndicate and sat on the strip

The syndicates can only do what they dobecause they control access to the mar-ket The Web notorious for its lack ofcontrol allows cartoonists to ldquosyndicaterdquotheir own work retain creative controland be more responsive to current

14th Systems Administration Conference (LISA 2000)NEW ORLEANS LOUISIANADECEMBER 3ndash8 2000ANNOUNCEMENTS


The first session started with the tradi-tional announcements from the programchairs Phil Scarr and Reacutemy Evard Philbegan with the following

There are over 1800 attendees mak-ing this our biggest LISA conferenceever

There were 85 papers submitted weaccepted 36 (42)

The Proceedings ran 378 pages Over 1800 messages were sent to

and from the program chairs to planthe conference

Sixty-three percent of the programcommittee changed jobs betweenLISA 1999 and LISA 2000

Thanks go to the program committeeinvited talks chairs network track coor-dinators security track coordinatorsguru coordinator WIPs coordinator thepaper readers Ellie Young and theUSENIX staff Rob Kolstad for his workin typesetting the Proceedings LyndaMcGinley for the terminal room and allthe vendors mentioned in the conferencedirectory

Reacutemy Evard then announced the bestpaper awards

System Honorable MentionmdashldquoDeployme Tellmersquos Package Manage-ment and Deploymentrdquo by Kyle Oppen-heim and Patrick McCormick of TellmeNetworks

Best Papers (tie)mdashldquoPeep (The NetworkAuralizer) Monitoring Your Networkwith Soundrdquo by Michael Gilfix and AlvaCouch of Tufts University and ldquoTracingAnonymous Packets to Their Approxi-mate Sourcerdquo by Hal Burch at Carnegie-Mellon University and Bill Cheswick of

C

ON

FERE

NC

ERE

PORT







INVITED TALKS


STOCK OPTIONS










SAGE UPDATE












THE DIGITAL HOUSE

Lorette Cheswick





6 Vol 26 No 2 login







intercom





Steve Romig OSU-IRT











C

ON

FERE

NC

ERE

PORT

S









ENVIRONMENT

















INTERNETS














8 Vol 26 No 2 login





Technologies









Steven Levine SGI







C

ON

FERE

NC

ERE

PORT


REFEREED PAPERS














MATURITY








Institute






PLACE



























SYSTEM


Board






AND SCRIPTS OH MY







C

ON

FERE

NC

ERE

PORT





University














UNDER X








SHARP









UDP Say no more




















SESSION 1984









MANAGEMENT




C

ON

FERE

NC

ERE

PORT















over time













APPRENTICE













Free Networks











URN


ITEP















C

ON

FERE

NC

ERE

PORT















DISTRIBUTION


University







DISTRIBUTION










SYSTEM


Technologies






































C

ON

FERE

NC

ERE

PORT

S





Research









NETWORK TRACK


YOUR NETWORK






















ANALYSIS


Corp




FOR IP NETWORKS
















Laboratory




CISCO NETFLOW LOGS




C

ON

FERE

NC

ERE

PORT





Madison























SECURITY TRACK



OTHER COPS


Computer Security

















C

ON

FERE

NC

ERE

PORT




Corporation














THE DOOR



APPROXIMATE SOURCE












Flight Center










SECURITY




Philosophy



YASSPrsquos project


tcpd+rpcbind





SECURITY EXAMINER











SECURITY




Communications Inc






WORKSHOP SERIES




PEAK Inc



C

ON

FERE

NC

ERE

PORT







WORKSHOP 3 METALISA




Staying Technical









Leaving Gracefully








































C

ON

FERE

NC

ERE

PORT




















































Tufts University





















WORKSHOP 9 TAXONOMY





C

ON

FERE

NC

ERE

PORT





BOFS

FREEBSD



















BOF NETBACKUP





POSTMASTER


Consulting





Open Source 812



rity add-ons

Open Source 9x







release 25weeks ago

SMI















UNIX ON HANDHELDS





C

ON

FERE

NC

ERE

PORT







INVITED TALKS


STOCK OPTIONS










SAGE UPDATE












THE DIGITAL HOUSE

Lorette Cheswick





6 Vol 26 No 2 login







intercom





Steve Romig OSU-IRT











C

ON

FERE

NC

ERE

PORT

S









ENVIRONMENT

















INTERNETS














8 Vol 26 No 2 login





Technologies









Steven Levine SGI







C

ON

FERE

NC

ERE

PORT


REFEREED PAPERS














MATURITY








Institute






PLACE



























SYSTEM


Board






AND SCRIPTS OH MY







C

ON

FERE

NC

ERE

PORT





University














UNDER X








SHARP









UDP Say no more




















SESSION 1984









MANAGEMENT




C

ON

FERE

NC

ERE

PORT















over time













APPRENTICE













Free Networks











URN


ITEP















C

ON

FERE

NC

ERE

PORT















DISTRIBUTION


University







DISTRIBUTION










SYSTEM


Technologies






































C

ON

FERE

NC

ERE

PORT

S





Research









NETWORK TRACK


YOUR NETWORK






















ANALYSIS


Corp




FOR IP NETWORKS
















Laboratory




CISCO NETFLOW LOGS




C

ON

FERE

NC

ERE

PORT





Madison























SECURITY TRACK



OTHER COPS


Computer Security

















C

ON

FERE

NC

ERE

PORT




Corporation














THE DOOR



APPROXIMATE SOURCE












Flight Center










SECURITY




Philosophy



YASSPrsquos project


tcpd+rpcbind





SECURITY EXAMINER











SECURITY




Communications Inc






WORKSHOP SERIES




PEAK Inc



C

ON

FERE

NC

ERE

PORT







WORKSHOP 3 METALISA




Staying Technical









Leaving Gracefully








































C

ON

FERE

NC

ERE

PORT




















































Tufts University





















WORKSHOP 9 TAXONOMY





C

ON

FERE

NC

ERE

PORT





BOFS

FREEBSD



















BOF NETBACKUP





POSTMASTER


Consulting





Open Source 812



rity add-ons

Open Source 9x







release 25weeks ago

SMI















UNIX ON HANDHELDS







THE DIGITAL HOUSE

Lorette Cheswick





6 Vol 26 No 2 login







intercom





Steve Romig OSU-IRT











C

ON

FERE

NC

ERE

PORT

S









ENVIRONMENT

















INTERNETS














8 Vol 26 No 2 login





Technologies









Steven Levine SGI







C

ON

FERE

NC

ERE

PORT


REFEREED PAPERS














MATURITY








Institute






PLACE



























SYSTEM


Board






AND SCRIPTS OH MY







C

ON

FERE

NC

ERE

PORT





University














UNDER X








SHARP









UDP Say no more




















SESSION 1984









MANAGEMENT




C

ON

FERE

NC

ERE

PORT















over time













APPRENTICE













Free Networks











URN


ITEP















C

ON

FERE

NC

ERE

PORT















DISTRIBUTION


University







DISTRIBUTION










SYSTEM


Technologies






































C

ON

FERE

NC

ERE

PORT

S





Research









NETWORK TRACK


YOUR NETWORK






















ANALYSIS


Corp




FOR IP NETWORKS
















Laboratory




CISCO NETFLOW LOGS




C

ON

FERE

NC

ERE

PORT





Madison























SECURITY TRACK



OTHER COPS


Computer Security

















C

ON

FERE

NC

ERE

PORT




Corporation














THE DOOR



APPROXIMATE SOURCE












Flight Center










SECURITY




Philosophy



YASSPrsquos project


tcpd+rpcbind





SECURITY EXAMINER











SECURITY




Communications Inc






WORKSHOP SERIES




PEAK Inc



C

ON

FERE

NC

ERE

PORT







WORKSHOP 3 METALISA




Staying Technical









Leaving Gracefully








































C

ON

FERE

NC

ERE

PORT




















































Tufts University





















WORKSHOP 9 TAXONOMY





C

ON

FERE

NC

ERE

PORT





BOFS

FREEBSD



















BOF NETBACKUP





POSTMASTER


Consulting





Open Source 812



rity add-ons

Open Source 9x







release 25weeks ago

SMI















UNIX ON HANDHELDS





C

ON

FERE

NC

ERE

PORT

S









ENVIRONMENT

















INTERNETS














8 Vol 26 No 2 login





Technologies









Steven Levine SGI







C

ON

FERE

NC

ERE

PORT


REFEREED PAPERS














MATURITY








Institute






PLACE



























SYSTEM


Board






AND SCRIPTS OH MY







C

ON

FERE

NC

ERE

PORT





University














UNDER X








SHARP









UDP Say no more




















SESSION 1984









MANAGEMENT




C

ON

FERE

NC

ERE

PORT















over time













APPRENTICE













Free Networks











URN


ITEP















C

ON

FERE

NC

ERE

PORT















DISTRIBUTION


University







DISTRIBUTION










SYSTEM


Technologies






































C

ON

FERE

NC

ERE

PORT

S





Research









NETWORK TRACK


YOUR NETWORK






















ANALYSIS


Corp




FOR IP NETWORKS
















Laboratory




CISCO NETFLOW LOGS




C

ON

FERE

NC

ERE

PORT





Madison























SECURITY TRACK



OTHER COPS


Computer Security

















C

ON

FERE

NC

ERE

PORT




Corporation














THE DOOR



APPROXIMATE SOURCE












Flight Center










SECURITY




Philosophy



YASSPrsquos project


tcpd+rpcbind





SECURITY EXAMINER











SECURITY




Communications Inc






WORKSHOP SERIES




PEAK Inc



C

ON

FERE

NC

ERE

PORT







WORKSHOP 3 METALISA




Staying Technical









Leaving Gracefully








































C

ON

FERE

NC

ERE

PORT




















































Tufts University





















WORKSHOP 9 TAXONOMY





C

ON

FERE

NC

ERE

PORT





BOFS

FREEBSD



















BOF NETBACKUP





POSTMASTER


Consulting





Open Source 812



rity add-ons

Open Source 9x







release 25weeks ago

SMI















UNIX ON HANDHELDS















8 Vol 26 No 2 login





Technologies









Steven Levine SGI







C

ON

FERE

NC

ERE

PORT


REFEREED PAPERS














MATURITY








Institute






PLACE



























SYSTEM


Board






AND SCRIPTS OH MY







C

ON

FERE

NC

ERE

PORT





University














UNDER X








SHARP









UDP Say no more




















SESSION 1984









MANAGEMENT




C

ON

FERE

NC

ERE

PORT















over time













APPRENTICE













Free Networks











URN


ITEP















C

ON

FERE

NC

ERE

PORT















DISTRIBUTION


University







DISTRIBUTION










SYSTEM


Technologies






































C

ON

FERE

NC

ERE

PORT

S





Research









NETWORK TRACK


YOUR NETWORK






















ANALYSIS


Corp




FOR IP NETWORKS
















Laboratory




CISCO NETFLOW LOGS




C

ON

FERE

NC

ERE

PORT





Madison























SECURITY TRACK



OTHER COPS


Computer Security

















C

ON

FERE

NC

ERE

PORT




Corporation














THE DOOR



APPROXIMATE SOURCE












Flight Center










SECURITY




Philosophy



YASSPrsquos project


tcpd+rpcbind





SECURITY EXAMINER











SECURITY




Communications Inc






WORKSHOP SERIES




PEAK Inc



C

ON

FERE

NC

ERE

PORT







WORKSHOP 3 METALISA




Staying Technical









Leaving Gracefully








































C

ON

FERE

NC

ERE

PORT




















































Tufts University





















WORKSHOP 9 TAXONOMY





C

ON

FERE

NC

ERE

PORT





BOFS

FREEBSD



















BOF NETBACKUP





POSTMASTER


Consulting





Open Source 812



rity add-ons

Open Source 9x







release 25weeks ago

SMI















UNIX ON HANDHELDS





C

ON

FERE

NC

ERE

PORT


REFEREED PAPERS














MATURITY








Institute






PLACE



























SYSTEM


Board






AND SCRIPTS OH MY







C

ON

FERE

NC

ERE

PORT





University














UNDER X








SHARP









UDP Say no more




















SESSION 1984









MANAGEMENT




C

ON

FERE

NC

ERE

PORT















over time













APPRENTICE













Free Networks











URN


ITEP















C

ON

FERE

NC

ERE

PORT















DISTRIBUTION


University







DISTRIBUTION










SYSTEM


Technologies






































C

ON

FERE

NC

ERE

PORT

S





Research









NETWORK TRACK


YOUR NETWORK






















ANALYSIS


Corp




FOR IP NETWORKS
















Laboratory




CISCO NETFLOW LOGS




C

ON

FERE

NC

ERE

PORT





Madison























SECURITY TRACK



OTHER COPS


Computer Security

















C

ON

FERE

NC

ERE

PORT




Corporation














THE DOOR



APPROXIMATE SOURCE












Flight Center










SECURITY




Philosophy



YASSPrsquos project


tcpd+rpcbind





SECURITY EXAMINER











SECURITY




Communications Inc






WORKSHOP SERIES




PEAK Inc



C

ON

FERE

NC

ERE

PORT







WORKSHOP 3 METALISA




Staying Technical









Leaving Gracefully








































C

ON

FERE

NC

ERE

PORT




















































Tufts University





















WORKSHOP 9 TAXONOMY





C

ON

FERE

NC

ERE

PORT





BOFS

FREEBSD



















BOF NETBACKUP





POSTMASTER


Consulting





Open Source 812



rity add-ons

Open Source 9x







release 25weeks ago

SMI















UNIX ON HANDHELDS






PLACE



























SYSTEM


Board






AND SCRIPTS OH MY







C

ON

FERE

NC

ERE

PORT





University














UNDER X








SHARP









UDP Say no more




















SESSION 1984









MANAGEMENT




C

ON

FERE

NC

ERE

PORT















over time













APPRENTICE













Free Networks











URN


ITEP















C

ON

FERE

NC

ERE

PORT















DISTRIBUTION


University







DISTRIBUTION










SYSTEM


Technologies






































C

ON

FERE

NC

ERE

PORT

S





Research









NETWORK TRACK


YOUR NETWORK






















ANALYSIS


Corp




FOR IP NETWORKS
















Laboratory




CISCO NETFLOW LOGS




C

ON

FERE

NC

ERE

PORT





Madison























SECURITY TRACK



OTHER COPS


Computer Security

















C

ON

FERE

NC

ERE

PORT




Corporation














THE DOOR



APPROXIMATE SOURCE












Flight Center










SECURITY




Philosophy



YASSPrsquos project


tcpd+rpcbind





SECURITY EXAMINER











SECURITY




Communications Inc






WORKSHOP SERIES




PEAK Inc



C

ON

FERE

NC

ERE

PORT







WORKSHOP 3 METALISA




Staying Technical









Leaving Gracefully








































C

ON

FERE

NC

ERE

PORT




















































Tufts University





















WORKSHOP 9 TAXONOMY





C

ON

FERE

NC

ERE

PORT





BOFS

FREEBSD



















BOF NETBACKUP





POSTMASTER


Consulting





Open Source 812



rity add-ons

Open Source 9x







release 25weeks ago

SMI















UNIX ON HANDHELDS





C

ON

FERE

NC

ERE

PORT





University














UNDER X








SHARP









UDP Say no more




















SESSION 1984









MANAGEMENT




C

ON

FERE

NC

ERE

PORT















over time













APPRENTICE













Free Networks











URN


ITEP















C

ON

FERE

NC

ERE

PORT















DISTRIBUTION


University







DISTRIBUTION










SYSTEM


Technologies






































C

ON

FERE

NC

ERE

PORT

S





Research









NETWORK TRACK


YOUR NETWORK






















ANALYSIS


Corp




FOR IP NETWORKS
















Laboratory




CISCO NETFLOW LOGS




C

ON

FERE

NC

ERE

PORT





Madison























SECURITY TRACK



OTHER COPS


Computer Security

















C

ON

FERE

NC

ERE

PORT




Corporation














THE DOOR



APPROXIMATE SOURCE












Flight Center










SECURITY




Philosophy



YASSPrsquos project


tcpd+rpcbind





SECURITY EXAMINER











SECURITY




Communications Inc






WORKSHOP SERIES




PEAK Inc



C

ON

FERE

NC

ERE

PORT







WORKSHOP 3 METALISA




Staying Technical









Leaving Gracefully








































C

ON

FERE

NC

ERE

PORT




















































Tufts University





















WORKSHOP 9 TAXONOMY





C

ON

FERE

NC

ERE

PORT





BOFS

FREEBSD



















BOF NETBACKUP





POSTMASTER


Consulting





Open Source 812



rity add-ons

Open Source 9x







release 25weeks ago

SMI















UNIX ON HANDHELDS






















SESSION 1984









MANAGEMENT




C

ON

FERE

NC

ERE

PORT















over time













APPRENTICE













Free Networks











URN


ITEP















C

ON

FERE

NC

ERE

PORT















DISTRIBUTION


University







DISTRIBUTION










SYSTEM


Technologies






































C

ON

FERE

NC

ERE

PORT

S





Research









NETWORK TRACK


YOUR NETWORK






















ANALYSIS


Corp




FOR IP NETWORKS
















Laboratory




CISCO NETFLOW LOGS




C

ON

FERE

NC

ERE

PORT





Madison























SECURITY TRACK



OTHER COPS


Computer Security

















C

ON

FERE

NC

ERE

PORT




Corporation














THE DOOR



APPROXIMATE SOURCE












Flight Center










SECURITY




Philosophy



YASSPrsquos project


tcpd+rpcbind





SECURITY EXAMINER











SECURITY




Communications Inc






WORKSHOP SERIES




PEAK Inc



C

ON

FERE

NC

ERE

PORT







WORKSHOP 3 METALISA




Staying Technical









Leaving Gracefully








































C

ON

FERE

NC

ERE

PORT




















































Tufts University





















WORKSHOP 9 TAXONOMY





C

ON

FERE

NC

ERE

PORT





BOFS

FREEBSD



















BOF NETBACKUP





POSTMASTER


Consulting





Open Source 812



rity add-ons

Open Source 9x







release 25weeks ago

SMI















UNIX ON HANDHELDS





C

ON

FERE

NC

ERE

PORT















over time













APPRENTICE













Free Networks











URN


ITEP















C

ON

FERE

NC

ERE

PORT















DISTRIBUTION


University







DISTRIBUTION










SYSTEM


Technologies






































C

ON

FERE

NC

ERE

PORT

S





Research









NETWORK TRACK


YOUR NETWORK






















ANALYSIS


Corp




FOR IP NETWORKS
















Laboratory




CISCO NETFLOW LOGS




C

ON

FERE

NC

ERE

PORT





Madison























SECURITY TRACK



OTHER COPS


Computer Security

















C

ON

FERE

NC

ERE

PORT




Corporation














THE DOOR



APPROXIMATE SOURCE












Flight Center










SECURITY




Philosophy



YASSPrsquos project


tcpd+rpcbind





SECURITY EXAMINER











SECURITY




Communications Inc






WORKSHOP SERIES




PEAK Inc



C

ON

FERE

NC

ERE

PORT







WORKSHOP 3 METALISA




Staying Technical









Leaving Gracefully








































C

ON

FERE

NC

ERE

PORT




















































Tufts University





















WORKSHOP 9 TAXONOMY





C

ON

FERE

NC

ERE

PORT





BOFS

FREEBSD



















BOF NETBACKUP





POSTMASTER


Consulting





Open Source 812



rity add-ons

Open Source 9x







release 25weeks ago

SMI















UNIX ON HANDHELDS










Free Networks











URN


ITEP















C

ON

FERE

NC

ERE

PORT















DISTRIBUTION


University







DISTRIBUTION










SYSTEM


Technologies






































C

ON

FERE

NC

ERE

PORT

S





Research









NETWORK TRACK


YOUR NETWORK






















ANALYSIS


Corp




FOR IP NETWORKS
















Laboratory




CISCO NETFLOW LOGS




C

ON

FERE

NC

ERE

PORT





Madison























SECURITY TRACK



OTHER COPS


Computer Security

















C

ON

FERE

NC

ERE

PORT




Corporation














THE DOOR



APPROXIMATE SOURCE












Flight Center










SECURITY




Philosophy



YASSPrsquos project


tcpd+rpcbind





SECURITY EXAMINER











SECURITY




Communications Inc






WORKSHOP SERIES




PEAK Inc



C

ON

FERE

NC

ERE

PORT







WORKSHOP 3 METALISA




Staying Technical









Leaving Gracefully








































C

ON

FERE

NC

ERE

PORT




















































Tufts University





















WORKSHOP 9 TAXONOMY





C

ON

FERE

NC

ERE

PORT





BOFS

FREEBSD



















BOF NETBACKUP





POSTMASTER


Consulting





Open Source 812



rity add-ons

Open Source 9x







release 25weeks ago

SMI















UNIX ON HANDHELDS





C

ON

FERE

NC

ERE

PORT















DISTRIBUTION


University







DISTRIBUTION










SYSTEM


Technologies






































C

ON

FERE

NC

ERE

PORT

S





Research









NETWORK TRACK


YOUR NETWORK






















ANALYSIS


Corp




FOR IP NETWORKS
















Laboratory




CISCO NETFLOW LOGS




C

ON

FERE

NC

ERE

PORT





Madison























SECURITY TRACK



OTHER COPS


Computer Security

















C

ON

FERE

NC

ERE

PORT




Corporation














THE DOOR



APPROXIMATE SOURCE












Flight Center










SECURITY




Philosophy



YASSPrsquos project


tcpd+rpcbind





SECURITY EXAMINER











SECURITY




Communications Inc






WORKSHOP SERIES




PEAK Inc



C

ON

FERE

NC

ERE

PORT







WORKSHOP 3 METALISA




Staying Technical









Leaving Gracefully








































C

ON

FERE

NC

ERE

PORT




















































Tufts University





















WORKSHOP 9 TAXONOMY





C

ON

FERE

NC

ERE

PORT





BOFS

FREEBSD



















BOF NETBACKUP





POSTMASTER


Consulting





Open Source 812



rity add-ons

Open Source 9x







release 25weeks ago

SMI















UNIX ON HANDHELDS








SYSTEM


Technologies






































C

ON

FERE

NC

ERE

PORT

S





Research









NETWORK TRACK


YOUR NETWORK






















ANALYSIS


Corp




FOR IP NETWORKS
















Laboratory




CISCO NETFLOW LOGS




C

ON

FERE

NC

ERE

PORT





Madison























SECURITY TRACK



OTHER COPS


Computer Security

















C

ON

FERE

NC

ERE

PORT




Corporation














THE DOOR



APPROXIMATE SOURCE












Flight Center










SECURITY




Philosophy



YASSPrsquos project


tcpd+rpcbind





SECURITY EXAMINER











SECURITY




Communications Inc






WORKSHOP SERIES




PEAK Inc



C

ON

FERE

NC

ERE

PORT







WORKSHOP 3 METALISA




Staying Technical









Leaving Gracefully








































C

ON

FERE

NC

ERE

PORT




















































Tufts University





















WORKSHOP 9 TAXONOMY





C

ON

FERE

NC

ERE

PORT





BOFS

FREEBSD



















BOF NETBACKUP





POSTMASTER


Consulting





Open Source 812



rity add-ons

Open Source 9x







release 25weeks ago

SMI















UNIX ON HANDHELDS





C

ON

FERE

NC

ERE

PORT

S





Research









NETWORK TRACK


YOUR NETWORK






















ANALYSIS


Corp




FOR IP NETWORKS
















Laboratory




CISCO NETFLOW LOGS




C

ON

FERE

NC

ERE

PORT





Madison























SECURITY TRACK



OTHER COPS


Computer Security

















C

ON

FERE

NC

ERE

PORT




Corporation














THE DOOR



APPROXIMATE SOURCE












Flight Center










SECURITY




Philosophy



YASSPrsquos project


tcpd+rpcbind





SECURITY EXAMINER











SECURITY




Communications Inc






WORKSHOP SERIES




PEAK Inc



C

ON

FERE

NC

ERE

PORT







WORKSHOP 3 METALISA




Staying Technical









Leaving Gracefully








































C

ON

FERE

NC

ERE

PORT




















































Tufts University





















WORKSHOP 9 TAXONOMY





C

ON

FERE

NC

ERE

PORT





BOFS

FREEBSD



















BOF NETBACKUP





POSTMASTER


Consulting





Open Source 812



rity add-ons

Open Source 9x







release 25weeks ago

SMI















UNIX ON HANDHELDS











ANALYSIS


Corp




FOR IP NETWORKS
















Laboratory




CISCO NETFLOW LOGS




C

ON

FERE

NC

ERE

PORT





Madison























SECURITY TRACK



OTHER COPS


Computer Security

















C

ON

FERE

NC

ERE

PORT




Corporation














THE DOOR



APPROXIMATE SOURCE












Flight Center










SECURITY




Philosophy



YASSPrsquos project


tcpd+rpcbind





SECURITY EXAMINER











SECURITY




Communications Inc






WORKSHOP SERIES




PEAK Inc



C

ON

FERE

NC

ERE

PORT







WORKSHOP 3 METALISA




Staying Technical









Leaving Gracefully








































C

ON

FERE

NC

ERE

PORT




















































Tufts University





















WORKSHOP 9 TAXONOMY





C

ON

FERE

NC

ERE

PORT





BOFS

FREEBSD



















BOF NETBACKUP





POSTMASTER


Consulting





Open Source 812



rity add-ons

Open Source 9x







release 25weeks ago

SMI















UNIX ON HANDHELDS





C

ON

FERE

NC

ERE

PORT





Madison























SECURITY TRACK



OTHER COPS


Computer Security

















C

ON

FERE

NC

ERE

PORT




Corporation














THE DOOR



APPROXIMATE SOURCE












Flight Center










SECURITY




Philosophy



YASSPrsquos project


tcpd+rpcbind





SECURITY EXAMINER











SECURITY




Communications Inc






WORKSHOP SERIES




PEAK Inc



C

ON

FERE

NC

ERE

PORT







WORKSHOP 3 METALISA




Staying Technical









Leaving Gracefully








































C

ON

FERE

NC

ERE

PORT




















































Tufts University





















WORKSHOP 9 TAXONOMY





C

ON

FERE

NC

ERE

PORT





BOFS

FREEBSD



















BOF NETBACKUP





POSTMASTER


Consulting





Open Source 812



rity add-ons

Open Source 9x







release 25weeks ago

SMI















UNIX ON HANDHELDS








SECURITY TRACK



OTHER COPS


Computer Security

















C

ON

FERE

NC

ERE

PORT




Corporation














THE DOOR



APPROXIMATE SOURCE












Flight Center










SECURITY




Philosophy



YASSPrsquos project


tcpd+rpcbind





SECURITY EXAMINER











SECURITY




Communications Inc






WORKSHOP SERIES




PEAK Inc



C

ON

FERE

NC

ERE

PORT







WORKSHOP 3 METALISA




Staying Technical









Leaving Gracefully








































C

ON

FERE

NC

ERE

PORT




















































Tufts University





















WORKSHOP 9 TAXONOMY





C

ON

FERE

NC

ERE

PORT





BOFS

FREEBSD



















BOF NETBACKUP





POSTMASTER


Consulting





Open Source 812



rity add-ons

Open Source 9x







release 25weeks ago

SMI















UNIX ON HANDHELDS





C

ON

FERE

NC

ERE

PORT




Corporation














THE DOOR



APPROXIMATE SOURCE












Flight Center










SECURITY




Philosophy



YASSPrsquos project


tcpd+rpcbind





SECURITY EXAMINER











SECURITY




Communications Inc






WORKSHOP SERIES




PEAK Inc



C

ON

FERE

NC

ERE

PORT







WORKSHOP 3 METALISA




Staying Technical









Leaving Gracefully








































C

ON

FERE

NC

ERE

PORT




















































Tufts University





















WORKSHOP 9 TAXONOMY





C

ON

FERE

NC

ERE

PORT





BOFS

FREEBSD



















BOF NETBACKUP





POSTMASTER


Consulting





Open Source 812



rity add-ons

Open Source 9x







release 25weeks ago

SMI















UNIX ON HANDHELDS









Flight Center










SECURITY




Philosophy



YASSPrsquos project


tcpd+rpcbind





SECURITY EXAMINER











SECURITY




Communications Inc






WORKSHOP SERIES




PEAK Inc



C

ON

FERE

NC

ERE

PORT







WORKSHOP 3 METALISA




Staying Technical









Leaving Gracefully








































C

ON

FERE

NC

ERE

PORT




















































Tufts University





















WORKSHOP 9 TAXONOMY





C

ON

FERE

NC

ERE

PORT





BOFS

FREEBSD



















BOF NETBACKUP





POSTMASTER


Consulting





Open Source 812



rity add-ons

Open Source 9x







release 25weeks ago

SMI















UNIX ON HANDHELDS





C

ON

FERE

NC

ERE

PORT







WORKSHOP 3 METALISA




Staying Technical









Leaving Gracefully








































C

ON

FERE

NC

ERE

PORT




















































Tufts University





















WORKSHOP 9 TAXONOMY





C

ON

FERE

NC

ERE

PORT





BOFS

FREEBSD



















BOF NETBACKUP





POSTMASTER


Consulting





Open Source 812



rity add-ons

Open Source 9x







release 25weeks ago

SMI















UNIX ON HANDHELDS





































C

ON

FERE

NC

ERE

PORT




















































Tufts University





















WORKSHOP 9 TAXONOMY





C

ON

FERE

NC

ERE

PORT





BOFS

FREEBSD



















BOF NETBACKUP





POSTMASTER


Consulting





Open Source 812



rity add-ons

Open Source 9x







release 25weeks ago

SMI















UNIX ON HANDHELDS










Tufts University





















WORKSHOP 9 TAXONOMY





C

ON

FERE

NC

ERE

PORT





BOFS

FREEBSD



















BOF NETBACKUP





POSTMASTER


Consulting





Open Source 812



rity add-ons

Open Source 9x







release 25weeks ago

SMI















UNIX ON HANDHELDS





C

ON

FERE

NC

ERE

PORT





BOFS

FREEBSD



















BOF NETBACKUP





POSTMASTER


Consulting





Open Source 812



rity add-ons

Open Source 9x







release 25weeks ago

SMI















UNIX ON HANDHELDS





POSTMASTER


Consulting





Open Source 812



rity add-ons

Open Source 9x







release 25weeks ago

SMI















UNIX ON HANDHELDS





inside - usenix.orgthe magazine of usenix & sage april 2001 • volume 26 • number 2 {inside:...

Documents