agent technology

Agent Technology:What Is It and Why Do We Care?

by James Odell, Senior Consultant,Cutter Consortium

Agent technology is now necessary to reduce costs; to improve

efficiency and effectiveness; and to support the requirements

of individuals, groups, companies, and universities as they

collaborate globally. More importantly, this type of technology

enables us to create and support a whole class of IT applications

and approaches that we previously could not have developed.

In this Executive Report, we discuss what agent technology

is as well as how and where it is currently being used in

various industries.

Enterprise Architecture

Vol. 10, No. 3

About Cutter ConsortiumCutter Consortium is a unique IT advisory firm, comprising a group of more than 150 internationally recognized experts who have come together to offer content,consulting and training to our clients. These experts are committed to delivering top-level, critical, and objective advice. They have done, and are doing, ground-breaking work in organizations worldwide, helping companies deal with issues inthe core areas of software development and agile project management, enterprisearchitecture, business technology trends and strategies, enterprise risk management,business intelligence, metrics, and sourcing.

Cutter delivers what no other IT research firm can: We give you Access to theExperts. You get practitioners’ points of view, derived from hands-on experience withthe same critical issues you are facing, not the perspective of a desk-bound analystwho can only make predictions and observations on what’s happening in themarketplace. With Cutter Consortium, you get the best practices and lessons learnedfrom the world’s leading experts, experts who are implementing these techniques at companies like yours right now.

Cutter’s clients are able to tap into its expertise in a variety of formats includingprint and online advisory services and journals, mentoring, workshops, training,and consulting. And by customizing our information products and training/consultingservices, you get the solutions you need, while staying within your budget.

Cutter Consortium’s philosophy is that there is no single right solution for allenterprises, or all departments within one enterprise, or even all projects within adepartment. Cutter believes that the complexity of the business technology issuesconfronting corporations today demands multiple detailed perspectives from which acompany can view its opportunities and risks in order to make the right strategic andtactical decisions. The simplistic pronouncements other analyst firms make do nottake into account the unique situation of each organization. This is another reason topresent the several sides to each issue: to enable clients to determine the course ofaction that best fits their unique situation.

For more information, contact Cutter Consortium at +1 781 648 8700 [email protected].

Cutter Business Technology Council

Access to the

Experts

Rob Austin Tom DeMarco Christine Davis Lynne Ellyn Jim Highsmith Tim Lister Lou Mazzucchelli Ken Orr Sheleen Quish Ed Yourdon

Centralizing a corporation wasonce considered an efficient wayto run an enterprise. Decisionsand information processingoccurred in an orderly, top-down,hierarchical manner. However, weare no longer in the era of main-frame computing, when bothcompanies and applications weretypically command-and-control-oriented and organized in verticalsilos. With the combination of theInternet, fiber optics, and PCs, thebusiness and technology playingfield has been flattened. No longerprimarily top-down, it has shiftedmore to a side-by-side approach— as individuals, small groups,and organizations interact aroundthe world. Furthermore, globaliza-tion and changes in technologyare causing today’s market tobe in a state of constant flux.Companies that cannot adapt fast

enough to thrive in new marketswill be left behind.

Agent technology is a primaryenabler to support this new era.In fact, without it, our currenttechnology will not scale to sup-port the ever-increasing globalinteraction. In response, manycompanies are now buildingagent-based systems. These sys-tems employ agents that can dis-tribute functionality across a vastcomputing network. Furthermore,agents not only adapt to theirenvironment but also evolve bylearning from the environment.In short, they are the ultimate indistributed computing. Such anapproach prepares enterprisesfor an increasingly complex mar-ketplace and enables them torespond rapidly to change.

The biggest breakthrough withagents is that they are an evolu-tion of existing technologies. Whatis revolutionary is the way wethink about and use agents todesign IT systems. They are beingbuilt from today’s technology andwill work together with today’stechnology. While agents, objects,relational databases, legacysystems, service-oriented archi-tectures (SOAs), event-drivenapproaches, and so on, eachhave their own niche, togetherthey can orchestrate rich systemsthat none of these technologiescould provide alone.

In this Executive Report, we firstdiscuss how agent technologies— including ant-based systems —are improving operations in dif-ferent industry sectors. We thenexplore agent technology in more

Agent Technology:What Is It and Why Do We Care?

ENTERPRISE ARCHITECTUREADVISORY SERVICEExecutive Report, Vol. 10, No. 3

by James Odell, Senior Consultant, Cutter Consortium

VOL. 10, NO. 3 www.cutter.com

22 ENTERPRISE ARCHITECTURE ADVISORY SERVICE

detail by defining it, examining itsinternal architecture and proper-ties, and understanding how vari-ous agent-based systems adaptand interact within environments.

WHY SHOULD WE CARE ABOUT AGENT TECHNOLOGY?

First, An Example fromEveryday Life

Living creatures and plants canproduce complex everyday

phenomena (e.g., ant colonies,traffic jams, stock markets, andprairie ecosystems). Similarly,automated entities — calledagents — can enable supply chainsystems, planning and schedulingapplications, Web services, and soon. Such agents need not be com-plicated. For example, the antcolony simulation of StarLogo1

(see Figure 1) uses softwareagents, where each ant followsthree simple rules:

1. Wander randomly.

2. If food is found, take a pieceback to the colony and leavea trail of pheromones, whichevaporates over time; then goback to rule one.

3. If a pheromone trail is found,follow it to the food and thengo back to rule two.

In Figure 1a, the “ant” agentsare just emerging from the anthillto begin their random walk.Eventually, an ant discovers afood source and returns some tothe colony, leaving a trail of evap-orative pheromones, as shown inFigure 1b. Figure 1c shows the antcolony well underway in retrievingthe food. Lastly, Figure 1d depictstwo depleted food sources andone that is exhausted altogether.

Ant-Like Agents at British Telecomand HP Labs

The ant colony is organizedwithout an organizer, coordinatedwithout a coordinator. As it turnsout, this simple ant foraging tech-nique provides an inspiration totackling some difficult technologi-cal problems. While the insect’stotally random approach tolocating food may appear to beinefficient, the study of this tech-nique has led to a new areaof research known as “swarmintelligence” [2].

The Enterprise Architecture Advisory Service Executive Report is published by Cutter Consortium, 37 Broadway, Suite 1, Arlington, MA02474-5552, USA. Tel: +1 781 641 9876 or, within North America, +1 800 492 1650; Fax: +1 781 648 1950 or, within North America, +1 800 888 1816; E-mail: [email protected]; Web site: www.cutter.com. Group Publisher: Kara Letourneau, E-mail: [email protected]. Managing Editor:Cindy Swain, E-mail: [email protected]. ISSN: 1530-3462. ©2007 by Cutter Consortium. All rights reserved. Unauthorized reproduction in any form, including photocopying, faxing, image scanning, and downloading electronic copies, is against the law. For information about reprints and/or back issues of Cutter Consortium publications, call +1 781 648 8700 or e-mail [email protected].

1StarLogo software is available for free at http://education.mit.edu/starlogo.

a

Figure 1 — Snapshots from an ant colony simulation using software agents.

b

c d

©2007 CUTTER CONSORTIUM VOL. 10, NO. 3

EXECUTIVE REPORT 33

For example, a decade ago,British Telecom (BT) used swarmintelligence for inspiration tomake software systems simpler,cheaper, and more intelligent androbust. At the time, Chris Winter,then head of BT Future Systems,was concerned that the trend ofIT systems was toward centralcontrol. “Ants walk around at ran-dom and there’s no intelligentcontroller telling them what todo,” said Winter. “When theyfind food, they simply run back tothe nest laying a little pheromonetrail that other ants can smell. Therest of the colony need only fol-low that path to fetch the remain-ing food” [3]. In other words,the processing is distributed, notcentrally controlled.

Winter used ants initially toweed out system bugs. Today,in BT’s development network,little ant-like programs run aroundfinding problems. They leavepheromone-like time stamps thathighlight the fault and draw helpfrom other software. In otherwords, BT is using the ants as afirst step in self-healing systemssoftware. Many software compa-nies are now embedding ant-likeagents in systems software andmiddleware for similar reasons.

In another application, BT andHP Labs created the world’s firstant-based systems [9]. Both tele-phone networks and the Internettypically route connectionsthrough a number of intermediateswitching stations. When the

network is large, many possibleroutes exist for each connection.Using a centrally controlled rout-ing approach, however, scalesbadly and can lead to failure ofthe entire system. A decentralizedmechanism like the ant-basedapproach was found to both scalewell with the network size andavoid the possibility of a system-wide failure.

The approach here is to send ant-like agents periodicallybetween nodes. At each node,the agent updates the node’srouting table with information(“pheromone”) indicating howlong the journey took from itsorigin and which nodes the agentused along the way. The routingtable contains a list of the node’simmediate neighbors and proba-bilities associated with using thatneighbor as the next step on thejourney toward a target node inthe network. The fastest ants willhave a positive effect on the prob-ability scores of the nodes theyused, while slow ants will have anegative effect.

Network congestion is anotherproblem that was addressed byBT and HP. They considered con-gestion in a particular section ofthe network to be analogous tothe depletion of a food sourcenear an ant colony. Here, ant-likeagents would search for newroutes and dynamically updatethe virtual pheromone trailrecorded in routing tables.

Other Uses of Ant-Based Systems

In ant-based systems, an antcolony of a finite size searchescollectively for a good solution toa given optimization problem.The optimal solution can only befound through the global coopera-tion of all the colony’s ants, wherethe ants only communicate indi-rectly by adding pheromones tothe environment. The BT applica-tions above are only two exam-ples of where techniques startwith models of ant behavior andthen add things that are not pre-sent in the real world. However,there are many more [2, 8], someof which include:

Traveling salesman problem.A salesman must find the short-est route by which to visit agiven number of cities, eachcity exactly once.

Quadratic assignment prob-lem. This involves the problemof assigning facilities to loca-tions so that the costs of theassignment are minimized.

Job-shop scheduling prob-lem. For a set of machines anda set of jobs, operations mustbe assigned to time intervalsin such a way that (1) no twojobs are processed at the sametime on the same machine,and (2) the maximum of thecompletion times of all opera-tions is minimized.

Vehicle routing problem. Thisinvolves finding minimum costvehicle routes where (1) every

customer is visited exactlyonce by exactly one vehicle;(2) for every vehicle, the totaldemand does not exceed thevehicle capacity; (3) the totaltour length of each vehicledoes not exceed a given limit;and (4) every vehicle startsand ends its tour at the sameposition (the depot).

Beyond Ants

Individual ants are not verysophisticated insects. They havelimited memory and a largely ran-dom element to their behavior.However, acting as a collective,ants can perform complex taskswith reliability and consistency.Ant colonies are not the onlythings that work like this. Bee-hives, flocks of birds, freewaytraffic, national and globaleconomies, societies, andimmune systems are all exam-ples of patterns that are deter-mined predominantly by localcomponent interaction insteadof centralized authority. For ITapplications, this can includeorder processing, supply chain,shop-floor control, inventorymanagement, message routing,and management of multipledatabases. In other words, adecentralized approach shouldbe considered where local com-ponents also have control —instead of limiting system-designapproaches solely to the cen-trally organized one traditionallyemployed by IT. After all, if NewYork City can maintain a two-week supply of food with onlylocally made decisions, why can’t

a supply chain system perform ina similar manner?

Painting Trucks at General Motors

Traditionally, assembly line sched-ules are centrally developed andcontrolled. Any change in theschedule must be centrally recon-figured. When the line is smalland has few unplanned stop-pages, centrally controlledschedules work well. However,scheduling for most real-worldassembly lines can be a night-mare: work stations break down,personnel get sick, environmentalconditions are not always withinacceptable limits, products com-ing down the line have or acquireunexpected defects, and so on.

Dick Morley, a technology vision-ary and father of the programma-ble controller, swept away oldassembly line schedules anddeveloped a better system forpainting trucks at GM’s assemblyplant in Fort Wayne, Indiana.“How do I schedule the non-schedulable?” Morley wondered.“Trucks do not come down theline in order of their color, and fre-quently no paint booth is availablewith the correct color.” Morleyalso discovered that many of thepaint booths were typically brokendown or being repaired.

In his technique, the schedulingprogram interacts with eachpaint booth. Instead of assigningunpainted trucks to booths, GM’ssolution was to have the boothsbid on the paint jobs [4]. Toaccomplish this, each booth was

equipped with a simple softwareagent that was programmed tokeep its booth busy and bid oneach paint job. The amount ofthe booth’s bid was based onhow busy the booth was at themoment of bidding, whether ithad to change to a different paint,and whether the booth was func-tioning properly.

To coordinate the various bidsfor each paint job, a scheduleragent acts as a broker. For exam-ple, when a truck arrives to bepainted, the scheduler agent tellsthe booths, “I have a truck thatneeds to be painted red.” A vacantpaint booth already loaded withred paint will bid very high.However, a vacant booth with adifferent color would bid lowerbecause of the extra labor andtime to clean and reload the paintgun. A booth that has just startedto paint a truck, has broken down,or is otherwise less suited for thejob would bid even lower. Basedon the outcome of the biddingactivity, the scheduler assigns thetruck to the highest-bidding paintbooth.

In a top-down planned “push-through” world, if one booth mal-functioned, a centrally controlledsystem would require immediaterecomputing. With bottom-up,“pull-through” paint booth agents,other booths were ready to pick upthe bidding slack at a moment’snotice. This new design saved $1million dollars in nine months andreduced the lines of computercode from hundreds to four [4].



Morley and GM tackled a problemwhere centralized scheduling didnot work efficiently by adopting anagent-based approach where eachbooth acts on its own behalf usinga market-based bidding system.Even though the scheduler was acentralized element, it deferred todistributed booth agents. Agent-based solutions do not removecentralization; instead, they try tobalance it with distributed solu-tions wherever it makes sense.

Dynamic Scheduling

Current market trends are drivingorganizations from mass produc-tion (where the supplier tells thecustomer what to buy) to masscustomization (where the cus-tomer tells the supplier what toprovide). For small supplier orga-nizations where resources andrequirements are reasonably sta-ble, this does not present a prob-lem. However, in larger and morecomplex organizations, supportingcentrally managed operations ismore difficult. This is particularlytrue of large suppliers with volatileand demanding conditions thatinclude unscheduled resourcefailures, periodic surges in neworders, and changes in require-ments and priorities. Using agentshere can change the perspectiveto a more distributed approach,making the solution more scal-able, adaptable, and robust. Infact, many organizations havealready realized their limit and

can no longer scale using acentralized approach. Organiza-tions like DHL and Credit Suissehave found that an agent-basedapproach is their only optionfor successfully managing thepreviously unmanageable sizeand complexity within theirbusinesses.

At the heart of the agent-basedsolution, two primary forms ofagents were used to gain distrib-uted control:2

1. Process-based agents. Thesehave the knowledge of how tocombine resources and createproducts as part of a workflowin a supply chain.

2. Resource-based agents.These manage the capacity-constrained resources of thesystems, such as people, vehi-cles, tools, machinery, materi-als, and facilities.

Business Processes and Resources Using Agents

Figure 2 contains an exampleof a workflow diagram for aprocess that installs utility polesfor an electric company. Theround-cornered rectangles repre-sent processes, and the square-cornered rectangles are thoseresources required by theprocesses.

In business process management(BPM) systems, such activities

are usually centrally managed.By using agents, however, theprocess and resources can bemanaged in a distributed manner.For example, an Install Pole agentis a specialized entity that knowsits Install Pole process needs autility pole, pole-installer equip-ment, a person to operate thepole installer, and the prior com-pletion of a Dig Pole Hole process.

Such knowledge can be usedin several ways. For instance, arequester might be interested in anInstall Pole service but would liketo know its cost before the serviceis invoked (see Figure 3). Here, acall for proposal (CFP) would besent to an agent that provides sucha service.3 The agent would senda CFP to those resources and pre-ceding services required for theInstall Pole process. Each resourceagent would then calculate thecharge for its usage based on thetime and duration specified by therequester. If a resource is urgentlyneeded and is in short supply, thecost might be high; if not, the pricewould be low. For example, if therequester wants a utility pole intwo hours and only an expensiveprovider has one available, theprice would be higher than if therequester could wait for two daysand obtain it from a cheaper sup-plier. In contrast, the agent repre-senting pole installers might beable to provide a person imme-diately. In two days, however,


EXECUTIVE REPORT 55

2Several other types of agents were used as well (e.g., order agents, dispatch agents, and supervisor agents). However, the process and resourceagents are key to understanding the approach.3Agents are a useful way to extend SOAs.

they could all be busy on otherassignments but would do therequested job at double-time rates.

Once the resources haveresponded to the CFP with abid, the Install Pole agent talliesup the cost for the service as awhole and sends a consolidatedbid to the requester for the InstallPole service.

In other words, the Install Poleagent acts like a broker on behalfof the process. If the requesteraccepts the bid, the broker isnotified of the award and will con-firm (or decline) the award.

Agent Negotiation

The interaction protocol justdescribed is expressed using the

sequence diagram depicted inFigure 4. The requester is play-ing the role of customer and theprovider is playing the role ofsupplier. Interaction protocols areuseful because they define theexpected behavior between inter-acting agents — whether eithercan be a resource agent or aprocess agent.4

Agent Renegotiation

In the real world, suppliers canovercommit, have resourcefailures, and experience processdelays. Using a top-down, central-ized approach, the schedulechanges tend to be reoptimizedfrom a global level. Such atechnique would work in a smalloperation but would not scale toa large one. However, using anagent-based approach — which is



Transformer Install

Transformer

Test

Tester

Cable

Worker

Cable

Puller

Pull Cable

Cable

Install Pole

Utility

Pole

Dig Pole Hole

Pole Hole

Digger

Pole Hole

Worker

Pole

Installer

Installer

Worker

Figure 2 — Example of a business process where processes and resources are managed using agents. (Source: [7].)

Pole Installer Installer

Operator

Utility

Pole

Dig Pole Hole

CFP forPole InstallerEquipment

CFP forInstaller Operator

Worker

CFP for Utility Pole

CFP for Dig Pole Hole

Install Pole

CFP for

Install Pole

Figure 3 — A call for proposal (CFP) to the Install Pole agent results in a CFP to all of its resources and preceding operations. (Source: [7].)

4It should be noted that the notion of requester and provider agents are at the very heart of the W3C’s Web Services Architecture study [11].

distributed by nature — enablesmore dynamic scheduling byadapting to these unexpected situ-ations on an individual and localbasis, rather than a massive andglobal one.

For example, an important jobrequest is received for ResourceA, which is already allocated toJob 1 and Job 2. The agent repre-senting Resource A needs to inter-act with the process agents foreach job to determine whetherthe extra work can be accommo-dated. The agent for Resource Aasks the agent for the first step inJob 1 about any extra (or slack),time (see Figure 5). It turns outthat if Resource A takes on thenew job, it will delay Step 1 byfour hours. Now, Step 1’s agentcan ask Step 2 if a four-hour delaywould affect its processing. SinceStep 2 has lots of slack time, it canabsorb the delay without affectingthe overall schedule.

Resource A’s agent must nowexamine the remaining optionof changing Job 2 (see Figure 6).Step 1 would be delayed by threehours. The Step 1 agent then asksStep 2 if a three-hour delay wouldimpact its processing. The Step2 agent indicates that such aslippage would also cause athree-hour delay for it. Step 3has a similar result and thereforeits agent must ask the end cus-tomer if this slippage presents aproblem. In this example, thecustomer has a large penaltyclause built into the contract forlate shipments. In summary, Job 2


EXECUTIVE REPORT 77

Supplier Customer

Call for Proposal

Bid/(no bid)

Award/(no award)

Confirm/(decline)

Proceed/(abort)

Figure 4 — An interaction protocol for making a contract.

Resource A Resource B

Resource CResource D

Job 1

Process Step 1Process Step 2


Job 1 could be delayed by four hours.

I wonder what my customer (Process

Step 2) thinks about that?

I can handle a four-hour delay without

affecting my customer. Therefore, I

won't charge for such delay.

Figure 5 — Renegotiation example for Job 1 by process and resource agents. Thebar represents the total time allocated to perform the process; the segment withinthe bar indicates the actual amount of time needed by the process. (Source: [1].)

Resource A Resource C

Resource G

Job 2

Process Step 1


Job 2 could be delayed

by three hours. I wonder

what my customer will

think about that?

If my supplier is late by three hours,

I will be late by three hours. I wonder

what my customer will think about that?

If my supplier is late by three

hours, I will be late by three

hours. I wonder what my

customer will think about that?

This is high priority.

That will really cost you!

Figure 6 — Renegotiation example for Job 2 by process and resource agents. (Source: [1].)

would result in a penalty if modi-fied to handle the extra require-ment for Resource A. However,adapting Job 1 would enableResource A to be assigned to anew job without affecting theoverall schedule. All decisions inthis example were based on alocal optimization rather than aglobal one — enabling decisionsto be made in seconds instead ofhours or days and reducing costsas well as time to market.

Agents and Dynamic Scheduling

The agent-based approachdescribed above was developedwith support from the RockIsland Arsenal as part of the USDepartment of Defense’s DefenseAdvanced Research ProjectsAgency (DARPA) contract tobuild an agent-based, factory-scheduling prototype, namedAutonomous Agents for RockIsland Arsenal. In one benchmarktest using these techniques, inven-tory costs were cut by 47%, leadtimes were cut 59%, and sched-ule-reducible costs (such as over-time and inventory holdingcharges) were cut by 93% [4].

Since that time, agent-basedschedulers have been designedfor suppliers in many industrysectors: manufacturing, finance,energy, and transportation.NewVectors LLC (http://www.newvectors.net), MagentaTechnology (www.magenta-technology.com), and IntelligentAutomation, Inc. — IAI (www.i-a-i. com/view.asp?tid=7) —have all been active in this area.

Magenta Technology has devel-oped an agent-based systemthat dynamically schedules thedeployment of ships and cargo.IAI is developing a genericscheduler that rapidly generatestailored and optimized schedulingengines. Using their software, anend user who has only domainexpertise can define:

Components (e.g., resources,work center, parts, tasks/operation, and jobs) — arerequired for the schedulingapplication, using the genericscheduling engine builder(GenSEB) component libraries

Protocols (interactions/constraints between compo-nents) — use a standardizedrepresentation languagecalled an agent interactionprotocol (AIP)

Rules/policies — define theorder of interactions and thecontent of an interaction basedon a user-defined schedulingalgorithm

Other Agent-Based Uses

Some Common Applications

Many companies have alreadytested agent-based applications intheir research and developmentfacilities. Successful applica-tions are now used commerciallythroughout the world. Here is abrief overview of some of them.

Just-In-Time Delivery

When operating in a highlycompetitive market, delivering

just-in-time (JIT) products andservices is vital. SCA Packagingturned to an agent-based model-ing solution that explored differentstrategies for reducing inventorylevels without compromising itsdelivery commitments. An agent-based simulation developed byEurobios (www.eurobios.com)enabled the company to reduceinventory levels by 35% whilemaintaining its delivery commit-ments [6].

Insurance Claims Processing

Acklin B.V. (www.acklin.nl)developed an agent-based,international vehicle–claimsprocessing system for threeEuropean companies. The agentsystem ensures EU regulationsfor confidentiality and enables arobustness that can survivebroader system shutdowns andfailures. As a result, the total timefor client and claim identificationis reduced from six months totwo minutes [6].

Distribution Optimization

Delivery of goods and servicesbetween site and customer isa problem for many industries.Companies such as BT, DHL,and Air Liquide (www.airliquide.com)require transportation net-works that can flexibly adaptin real time to changing andunforeseen run-time conditions,fluctuations, and requirements.Not surprisingly, at least threecompanies provide agent-based solutions to this area:IAI, NuTech Software Solutions



(www.nutech.com), andWhitestein Technologies (www.whitestein.com).

Onsite Maintenance

Any industry that employs JITprocesses relies on the availabil-ity of manufacturing equipment.Faulty equipment can have cata-strophic consequences on pro-ductivity. In this context, bothpreventive maintenance and cor-rective maintenance become keyactivities. In the agent systemdeveloped by Oslo Software(www.oslo-software.com), itsautomotive customer handled150,000 pieces of equipment withtwo million maintenance opera-tions per year. The automotivecompany was originally usingthe maintenance module of awell-known enterprise resourceplanning (ERP) software vendor.While the ERP package couldprovide preventive-maintenanceschedules, it could not handle thecorrective-maintenance opera-tions. Here, team managers had todeal with exceptional corrective-maintenance operations using theless practical Excel format. Thislead to canceling an average of1,000 maintenance operationsevery month, increasing therisk of equipment failure. Themaintenance problem is complexbecause many pieces of equip-ment are involved and the skillsrequired of the personnel arevaried. New technologies areneeded to automate the mainte-nance process. These technol-ogies need to automate the

exception-handling process inorder to provide better support tobusiness users. Agent technology,by its unique exception-handlingcapability, provides the solution tothis problem.

Healthcare

By modeling the stakeholdersin primary care systems asagents, Calico Jack (www.calicojack.co.uk) has developeda way to integrate health systemsfor the Scottish Executive HealthDepartment.

Simulation

Since multiagent systems involvemany interacting agents, predict-ing the whole system’s successor failure is difficult. Defining thebehavior of each agent is onething; knowing what will emergefrom their joint interaction isanother.

Multiagent models can be usedto simulate the behavior of com-plex computer systems, includingmultiagent computer systems.Such simulation models canassist designers and developersof complex application systemsand provide guidance to softwareengineers. Agent-based simulationis also an important techniquethat offers strong models for rep-resenting complex and dynamicreal-world environments. Agentsystems simulating real-worlddomains may provide answersto certain physical or socialproblems that would be other-wise unobtainable due to their

complexity. Agent-basedsimulation spans many areas.For example, it can aid socialstructures and institutions indeveloping plausible explana-tions of observed phenomena;assisting in designing organiza-tional structures; and informingpolicy or managerial decisions.Agent-based simulation can helpmodel physical systems, includingintelligent buildings, traffic sys-tems, biological populations,and software systems of all types,including e-commerce and infor-mation management systems [6].The two most popular agentsimulators are Swarm (www.swarm.org) and CybelePro(www.cybelepro.com). Recently,CybelePro was used to simulateair transportation systems involv-ing flights, airports, terminal areas,controllers, and airline operationcenters with varying levels offidelity. Here, an individual simula-tion is comprised of tens of thou-sands of dynamically interactingagents configured to represent aconcept/scenario.

Collaborative Decision Makingand Distributed Control

As systems become increasinglydecentralized and autonomous,technology needs to support col-laborative, team-based decisionmaking and control. Agents canwork individually and sociallyin teams to aid and accomplishthis. In particular, IAI has devel-oped applications for team for-mation, cooperative sensing,tracking, monitoring, and trafficmanagement.


EXECUTIVE REPORT 99

Supply Chains and Logistics

Today’s supply networks needto be flexible, responsive, adap-tive, and able to cope with thevariability of demands. Tradi-tional supply chain and logisticsoptimization systems are notdesigned to handle volatility andcomplexity or to function in a real-time environment. Using anagent-based approach is now acommon solution for many com-panies. Software vendors utilizingthis type of approach includeMagenta Technology, IAI, andCougaar Software, Inc. (www.cougaarsoftware.com).

Autonomic Computing

In today’s global networks, weneed computer systems thatregulate themselves much in thesame way our autonomic nervoussystem regulates and protectsour bodies. Self-healing systemsrequire an agent-based approachrather than a centralized one.

Architectures and Ontologies —and Agents

In addition to specific applica-tions, the IT developer mustaddress current architecturalapproaches. Here is a briefoverview of those approachesparticularly appropriate for agents.

Ontologies

Agents require a common vocab-ulary and set of concepts to com-municate effectively with otheragents. Agent-system developers

have been adapted to use thesemantic Web language (OWL),the semantic Web rule language(SWRL), and the services ontol-ogy (OWL-S) for the specificationof agent systems. In addition,agents can be used to manageand maintain large distributedontologies as well as perform datamining functions. Since ontologiesare both syntactically and seman-tically defined, agents not onlyintelligently query, infer, and rea-son over ontologies, their behaviorcan also be guided by an ontology.These are the so-called ontology-driven approaches. Here, theagent code does not have to bemodified and recompiled forevery change. Instead, the ontol-ogy can be changed, resulting ina dynamic change in the agents’behavior based on the modifiedontology.

Peer-to-Peer

Peer-to-peer (P2P) applicationsdisplay agent-like characteristicsthat include both applying self-organization techniques thatensure continuous operation ofthe network and employing inter-action protocol designs to enforcecorrect behavior among interact-ing nodes. For example, commer-cial e-marketplace systems likeeBay include simple credit reputa-tion systems to reward sociallybeneficial behavior. As P2Psystems become more complex,increasing the use of agent tech-nologies will be appropriate.For example, the auction-basedmechanisms and negotiation

techniques used by agents couldbe used to enhance the level ofautomation of peers in popularapplications. Since complex P2Papplications are social in nature,they require increasingly sophisti-cated approaches to trust andreputation and to the applicationof social norms, rules, and struc-tures. Social simulation wouldbe particularly appropriate herein order to better understand thepopulation dynamics of indepen-dent agents [6].

Web Services and SOA

The W3C Web Services Archi-tecture study recommends anagent-based approach to usingSOAs — both on the servicerequester side and the serviceprovider side [11]. Agents actwith varying levels of autonomy,depending on environmentalconstraints and their ongoinginteractions. Because servicesare often best modeled asautonomous and heterogeneous,they can naturally be associatedwith agents. Agents are not apanacea. However, when appliedappropriately, they enable us to(1) define elaborate constraintson what services are willing tooffer, thereby enabling us todefine richer requirements forservice composition; (2) discovertrustworthy services; (3) negotiatewith external service providers;and (4) evaluate the complianceof service providers within theircontracts. They can also providea more natural, human-like way of interacting with service



requesters, as well as ensuringthe entire process is carried outeffectively and efficiently [10].

Verizon chose a decentralizedagent-based approach to SOArather than a centralized approachwhere all services are handledby a server or set of servers.Additionally, Verizon employsagents to support the subscription,management, and dashboardlayers of its SOA, using its ITWorkbench software. By theend of 2004, Verizon was handlingnearly three million service trans-actions per day.

Service composition aggregatesservices to create new functional-ity. Often, the composed function-ality would itself be exposed asa new service with a standardinterface. If this happens, a newservice could be composed thatwould intelligently guide theservice requester throughout thelifetime of a particular businessprocess. Such an intelligent ser-vice could be implemented as anagent. Agents also make it possi-ble to capture the interactionsamong services and to create newservices as subtle compositionsof others.

BPM

BPM has come to mean anintegrated collection of criticalprocess technology necessary tosupport the business process life-cycle (also called the BPM suite).The actual business process logiccan be centralized in one location,as opposed to being distributed

across and embedded within mul-tiple services. However, multiplebusiness processes can be usedto satisfy the service (see Figure7). For example, to request aproduct’s price, the computationcould vary depending uponlocation of order, time and dateordered, quantity, customer, andother items in the same order.

The DHL pricing system, forexample, had nearly 500 differentprocesses for pricing an orderbecause of the number and poten-tial combinations of variables. Tohandle this, DHL used agents tochoose the appropriate processes.In other words, each agent con-tained a process-execution enginethat would select and executethe appropriate process for eachpricing-query service request. If aproblem arose in the middle of aprocess, the agent would select analternate one. DHL found that anagent-based approach not onlyexecuted faster, but was easier tomaintain. Changes to a process orprocess rule (such as adding a fuelsurcharge) took hours rather thanweeks. In a situation like this, soft-ware from companies like OSLOcan be employed, which uses agraphic editor based on UML activ-ity diagrams — enabling a user tochange the process. Again, agentsare not always appropriate forcontrolling business-process logic.If the process is simple andstraightforward and it does notrequire frequent changes and pro-cessing robustness, a languagelike BPEL could be used. However,for complex processes that involve

various operating conditionsand require frequent and timelychanges, agents should be used.

Event-Driven Architectures

Real-time response is often criti-cal to customer satisfaction.Abnormal events or a combina-tion of events can have a signifi-cant impact on an organization.More and more companies,such as IAI and Rhysome (www.rhysome.com), are using agentsalong with software and physicalsensors to detect changes in thebusiness environment. (Examplesinclude RFID tags for retail supplychain optimization, medicalmonitors, physical sensors thatdetect changes in air quality, andelectronic data capture tools forpatient trials at pharmaceuticalcompanies.) Companies thatderive the greatest advantagesfrom agent-based, event-drivenarchitectures (EDAs) have the fol-lowing characteristics: (1) largeand heterogeneous environments,(2) information that changesconstantly in various ways,


EXECUTIVE REPORT 1111

Service

Figure 7 — Services can be carried out by many different

business processes depending on business conditions and rules.

(3) complex exceptions and statechanges in real time, and (4) theneed to deliver and respondappropriately to that information.

Integrating SOA, BPM, and EDA

These last three architecturalapproaches can be consideredseparately for including agenttechnology. Additionally, agenttechnology can be used to linkbusiness processes and servicesto facilitate a single architecturethat integrates SOA, BPM, and EDA(see Figure 8).

Why Use Agent-Based Technology?

Thus far, we’ve examined a sam-pling of uses for agent-basedsystems — happening now withinthe IT community. Given that anynew technology is disruptive,why did these early adopter com-panies decide to use agent tech-nology? The reasons can besummarized by one or more ofthe following benefits:

Faster ROI

Lower maintenance

Higher productivity

Leverage of existinginfrastructure

Reuse of processes andservices

Foundation for future projects

Reduction of time to market

Increased agility to respondto business needs

Is an agent-based approach usefulfor every application and usage?Does it always provide the bene-fits listed above? Certainly not. Ifyour business is predictable andstable and if your processes arecentralized and scalable for theforeseeable future, adopting anagent-based approach is notnecessary. However, for thoseapplications that must supportcomplexity and change in ascalable and timely manner,agents will likely be a necessarytechnology.

In the typical organization, bothof these situations probably coex-ist in one form or another. Savvyorganizations will respond with a

mixture of technologies: object-orientated (OO), relational, andagent-based. OO and relationaltechnologies enable a top-downand centralized solution to busi-ness application. Agents provideone more tool that conventionalIT shops do not have — a bottom-up and distributed app-roach. The real benefit comeswhen an organization can choosethe appropriate mix of technolo-gies for a given application, whichprovides a balance of both thecentralized and distributedapproaches.

It is one thing to say that agentscan provide a helpful, distributiveapproach to software applications;it is quite another to understandwhat that means and entails. Thenext section begins that journeyby answering the question “Whatis an agent?”

WHAT IS AN AGENT?

Conventional objects can bethought of as passive, becausethey wait for a message beforeperforming an operation. Onceinvoked, they execute theirmethod and go back to “sleep”until the next message. A currenttrend in many systems is to designobjects that both react to eventsin their environment and areproactive. In UML 2.0, these areknown as active objects; in theagent community, they are knownas agents. Whether they arecalled active objects or agents,this new direction is going tochange radically how we designsystems.



Agents

Services

Business Process

Orchestration

Choreography

Business

Process

Management

(BPM)

Event-Driven

Architecture

(EDA)

Service-Oriented

Architecture

(SOA)

Figure 8 — Agents can be used to integrate SOA, BPM, and EDA.



Autonomy

Interactivity

Adaptivity

multiagent society

simple agent

interaction

more externally controlled

more self-governing

learning/

evolving

reactive

only

The Basic Properties of Agents

An agent can be a person, amachine, a piece of software, ora variety of other things. The basicdictionary definition of agent is“something that acts.” However,for developing business and ITsystems, such a definition istoo general. While an industry-standard definition of agent hasnot yet emerged, most agree thatagents deployed for IT systemsare not useful without the follow-ing three important properties:

1. Autonomous. The agent iscapable of acting without directexternal intervention. It hassome degree of control over itsinternal state and actions basedon its own experiences.

2. Interactive. The agent commu-nicates with the environmentand other agents.

3. Adaptive. The agent is capableof responding to other agentsand/or its environment. Anagent can modify its behaviorbased on its experience.

Based on this approach, a basicworking definition for an agent is“an autonomous entity that canadapt to and interact with itsenvironment.”

Agents are commonly regardedas autonomous entities becausethey can be thought of as havingtheir own set of internal responsi-bilities and processing. For exam-ple, each ant in Figure 1 has itsown self-contained processingthat enables searching for and

gathering of food without anyexternal choreography.

Agents are interactive entitiesbecause they are capable ofexchanging rich forms of mes-sages with other entities in theirenvironment. These messagescan support requests for servicesand other kinds of resources, aswell as event detection and notifi-cation. They can be synchronousor asynchronous in nature. Theinteraction can also be conversa-tional in nature — negotiatingcontracts, marketplace-stylebidding, or simply making aquery. In the ant colony example,ants interact via the pheromonesthat they deposit in the environ-ment. The pheromones act asinformation signposts for otheragents, providing a simple yethighly effective means ofcommunication.

Lastly, agents can be thought ofas adaptive because they canreact to messages and eventsand then respond appropriately.In the example above, eachant adapts to its environment bycontinuing to wander randomly ifit fails to find food or pheromones.However, when pheromonesare detected, an ant reacts bychanging its behavior to track thepheromones to the food source.Once the food is found, the antagain adapts by picking up thefood and carrying it back to thecolony. If the food is moved, theadaptive ants will locate the newfood source and notify otherswhile bringing it to the colony.

Ants provide a good exampleof simple reactive adaptation.However, agents can also bedesigned to learn and evolve.

Overall, agents can be auton-omous, interactive, and adaptiveto some degree (see Figure 9). Itis not an all-or-nothing proposi-tion. In the next few sections,these three key agent propertieswill be discussed in more detail.

Agents Are Autonomous

Agents can be thought of asautonomous because eachis capable of governing itsown behavior to some extent.Autonomy is best characterizedby degrees. At one extreme, anagent could be completely self-governing and self-contained.However, an agent that does notrequire the resources of or inter-action with other entities is veryrare. Even a database accessor Web query agent requiresexternal resources. At the otherextreme, an agent that is barelyable to perform the simplest ofactions is impractical for the prag-matic system developer. Eventraditional objects have somedegree of self-contained lines ofcode within their methods.

Depending on the system devel-oper, an agent’s autonomy canbe designed to fit some placebetween these two extremes.For example, conventional objectsare not completely dependenton outside resources; they havesome state and behavior of theirown (see Figure 10). But they are

traditionally simple, which meansthat they are far from being com-pletely independent. Because sup-ply chain agents are able to reachcertain conclusions and makedecisions on their own, they havemore autonomy than objects. Bytheir very nature, supply chainagents require interactions withother entities to enable a fullyfunctional supply chain, sinceone agent cannot take care of anentire organization’s supply chain.More autonomous are manageragents because their role can

involve a high degree of internaldecision making though activitiessuch as monitoring and delegat-ing, which depend on outsideresources.

Two major aspects of an agent’sautonomy involve its capacity tobe dynamic and its ability to makedecisions. As depicted in Figure11, autonomy can be consideredon two axes. On one axis, thedynamic aspect of an agent’sautonomy can range from beingsimply passive in its action to

entirely proactive. On the otheraxis, an agent’s decision-makingaspect ranges from being limitedto simple decisions to being capa-ble of making complex decisions(see Figure 11).

Dynamic Autonomy

In addition to reacting to specificmethod invocations, agents canrespond to events within theirenvironment. Proactive agents willactually poll the environment forevents and other messages todetermine what action theyshould take. To compound this,in multiagent systems, agents canbe engaged in multiple parallelinteractions with each other —magnifying their dynamic nature.In short, an agent can decidewhen to say “go.”

As described earlier, the GM paintbooths had both reactive andproactive features. Informationabout an unpainted car or truckcoming down the line was postedin an automated form accessibleto all paint booths. When a paintbooth had nearly completed itscurrent job, it basically said,“Hmmm, I’m running out of work,I’ll look over at the jobs posted.”As stated earlier, if the boothwas applying the color of paintrequired by an upcoming job, itwould bid more for the job thanwould a booth having a differentcolor. Other bidding criteria couldinclude how easy or important thejob was. In other words, the boothwas reactive in whether to bid ona paint job, but proactive in that it



Autonomy

Interactivity

Adaptivity

multiagent society

simple agent

interaction

more externally controlled

more self-governing

learning/

evolving

reactive

only

Figure 9 — Agents have various degrees of autonomy, interactivity, and adaptivity.

?ObjectsSupply Chain

Agents Manager Agents

Complete

Autonomy

Complete

Dependence

lower

dependency

on other entities

higher

dependency

on other entities

Figure 10 — Degrees of autonomy.

would check the paint job list anddetermine what to do.

Objects, on the other hand, areconventionally passive — withtheir methods being invokedunder a caller’s thread of control.The term “autonomy” barelyapplies to an entity whose invoca-tion depends solely on other com-ponents in the system. However,UML and Java have recently intro-duced event-listener frameworksand other mechanisms for allow-ing objects to be more active. Inother words, objects now havesome of the dynamic capabilityof agents.

Decision Autonomy

Agents can be designed to makea few simple decisions. However,the more decisions an agent iscapable of making and the morecomplex they are, the moreautonomous the agent appears.A completely autonomous agentwill do whatever it wishes.

For example, an ant that is wan-dering around looking for foodcan appear to be taking a randomwalk. However, once pheromonesor food are detected, the choicesfor its behavior are predeter-mined. In contrast, the GM paintbooth can decide on how muchto bid based on the booths abilityto efficiently take on a new paintjob. Certainly, the choices are lim-ited for a paint booth, but the anthas no choice on how to actwhen food is found. In contrast,shopping agents can have very

complex criteria for selecting agift for a specific individual. Infact, the agent might return emptyhanded because it decided nogifts were appropriate. A contract-negotiation agent may decide thecontract is not worth pursuing fora number of reasons. In otherwords, the agent can also say“no” to performing a requestedservice.

Conventional objects certainlyhave the ability to make decisions.However, the typical usage anddirect support with OO languagestend toward a less complicatedapproach to decision making. Forinstance, when a message is sentto an object, the method is alwaysinvoked. The contract-negotiationagent could refuse to invoke arequested bid method; the objectcannot. Yes, an object may deter-mine whether or not to process

the message and how to respondif it does. In common practice,however, the refusal of an objectto execute its method is typicallyconsidered an error situation.With agents, this is not the case.Manager and quality assuranceagents are good examples ofagents that can say “no.”

Object classes are usuallydesigned to be highly predictablein order to facilitate buying andselling reusable components.Agents are commonly designedto determine their behavior basedon their individual goals andstates as well their contexts withinongoing conversations with otheragents. While OO implementa-tions can certainly be developedto include nondeterministicbehavior, this is common in agent-based thinking.



Decision Ability

Can Say “No”D

yn

am

ic A

bil

ity

/

Ca

n S

ay

“G

o”

Proactive

Reactive

Simple

decisions

Complex

decisions

Passive

Clock

Supply Chain

Agent

GM Paint

Booth

Shopping

Agent

Java/UML

Object

Ant

Object

Figure 11 — Two aspects of autonomy.

Agents Are Interactive

Interaction involves the ability tocommunicate with the environ-ment and other entities. It canbe expressed in degrees (seeFigure 12).

On one end of the scale, objectmessages (method invocation)can be seen as the most basicform of interaction. A more com-plex degree of interaction wouldinclude those agents that canreact to observable events withinthe environment. For example,food-gathering ants don’t invokemethods on each other; theirinteraction is indirect, throughdirect physical effects on theenvironment. Even more complexinteractions are found in multi-agent systems where agentsengage in multiple, parallel inter-actions with other agents. Here,agents begin to act as a society.

Finally, the ability to interactbecomes most complexwhen systems involving manyheterogeneous agents can coordi-nate through cooperative and/orcompetitive mechanisms (suchas negotiation and planning).

While we can conceive of anagent that cannot interact withanything outside of itself, the use-fulness of such an entity for devel-oping agent-based systems isquestionable.

Agents Are Adaptive

An agent is considered adaptiveif it can respond appropriatelyto other agents and/or its environ-ment to some degree. Auton-omous entities that fail to adaptrapidly to the changing worldbecome extinct: organisms die;companies go out of business.For an organization, adaptationenables the system to react effec-tively to changes in areas such asthe market and business environ-ment. In IT systems, adaptationenables systems to react appro-priately enabling, for example,system balancing, integrity assur-ance, and self-healing systems.When designed properly, the indi-vidual parts of the system can beempowered to change basedon environment and marketconditions.

Simple Reactivity

At a minimum, this means that anagent must be able to react to a

simple stimulus — to make adirect, predetermined responseto a particular event or environ-mental signal. Such a responseis usually expressed in “if-then”form. Thermostats, robotic sen-sors, and simple search bots fallinto this category.

From atoms to ants, the reactivemode is quite evident. A carbonatom has a rule that states ineffect: “If I am alone, I will onlybond with oxygen atoms.” Anant has a rule that if it finds food, itshould return the food to its colonywhile leaving a pheromone trail.

Rules

Beyond the simple reactive agentis the agent that can appear toreason by following chains ofrules. For example, agents canreact by making inferences andinclude patient diagnosis agentsand certain kinds of data miningagents. These inferences arecomputed by following a chainof inference rules.

Business process engines canuse a similar approach. Here,libraries of processes are main-tained, which indicate thecircumstances under which aparticular process might beappropriate. Each step within aprocess acts as a service request,invoking the business processengine to choose the appropriateset of steps that should be exe-cuted, based on the business con-text. For example, a request tocompute the sales task would



Objects AntsSupply Chain

Agents

simple complex

Method Invocation

Society Interaction

Figure 12 — Degrees of interaction.

trigger the engine to locate theappropriate set of steps neededfor a given location. The taxes inMichigan would be computed dif-ferently than in Brussels.

Rules do not change in and ofthemselves. Instead, change cancome through other mechanismssuch as learning and evolution.Without learning and evolution,ants and atoms are still quite ableto support complex “societies.”With learning and evolution, how-ever, the rules can be changedbased on experience — resultingin new and perhaps improvedresults.

Learning

Learning is change that occursduring the lifetime of an agent andcan take many forms. The mostcommon techniques enable rulesand decisions to be weightedbased on positive (or negative)reinforcement. For example, in abasic bidding system, a bid couldbe selected simply on the basis ofbid price. However, other consid-erations might also be appropriatesuch as the bidder’s ability todeliver its goods in the quantity,quality, and timeframe requested.Over time, a purchasing agentcan learn to choose from reliablevendor agents instead of justchoosing the lowest bid. If a ven-dor’s performance improves (ordeclines), the purchaser’s deci-sions are modified accordingly.In other words, the agent contin-ues to learn. Popular learning tech-

niques that employ reinforcementlearning include credit assign-ment, Bayesian and classifierrules, and neural networks.Examples of learning agentswould be agents that can approvecredit applications, analyzespeech, or recognize and track tar-gets.

Evolution

Evolution is change that occursover successive generations ofagents. A primary technique foragent evolution usually involvesgenetic algorithms and geneticprogramming. Agents can literallybe bred to fit specific purposes.For example, operation plans, cir-cuitry, and software programs canprove to be more optimal thanany product that a human canmake in a reasonable amountof time.

Other Agent Properties

In earlier sections, three key agentproperties were discussed: auton-omy, interactivity, and adaptivity.These properties are importantbecause agents deployed for ITsystems are not useful withoutthem. However, agents may pos-sess various combinations ofother properties that may beuseful — depending on theapplication requirements andthe agent designer. Here, agentsmay be:

Sociable — interaction markedby friendliness or pleasantsocial relations (i.e., the agent

is affable, companionable, orfriendly)

Mobile — able to transportitself from one environmentto another

Proxy — may act on behalf ofsomeone or something (i.e.,acting in the interest of, as arepresentative of, or for thebenefit of some other entity)

Intelligent — state is formal-ized by knowledge (i.e., beliefs,goals, plans, assumptions)and interacts with other agentsusing symbolic language

Rational — able to choose anaction based on internal goalsand the knowledge that a par-ticular action will bring it closerto its goals

Temporally continuous — is acontinuously running process

Credible — believable person-ality and emotional state

Transparent and accountable— must be transparent whenrequired, yet must provide a logof its activities upon demand

Coordinative — able to per-form some activity in a sharedenvironment with other agents;activities often coordinated viaplans, workflows, or someother process managementmechanism

Cooperative — able to coordi-nate with other agents toachieve a common purpose;nonantagonistic agents that



succeed or fail together (alsoknown as collaboration )

Competitive — able to coordi-nate with other agents, but thesuccess of one implies thefailure of others (opposite ofcooperative)

Rugged — able to deal witherrors and incomplete datarobustly

Trustworthy — adheres toLaws of Robotics and is truthful

WHAT GOES ON INSIDE AN AGENT?

As we discussed in the previoussection, agents are autonomousentities that can adapt to andinteract with their environment. To

accomplish this, agents must havesome means by which they canperceive their environment andact upon those perceptions (seeFigure 13).

Agent as a Black Box

In its simplest form, each agentcan be thought of as an interactiveblack-box process, where noindication of an agent’s internalstructure and behavior is givenother than that which is visiblefrom outside the agent. Figure 14depicts this by illustrating thatagents have input from, andoutput to, the environment. Anagent’s environment can bethought of as everything outside ofthe agent. For robotic agents, theenvironment would consist of the

terrain, the laws of physics, andother agents. For software agents,the environment would consist ofthe hosting platform, its operatingsystems and supporting middle-ware, and other agents.

Input can be whatever the agent“perceives,” such as messages,requests, commands, and events.Based on this input, the agent canchoose how to act on it, whichtypically results in output from theagent such as communicatingwith, or making a change to, itsenvironment. For instance, anInventory agent could receivea message that requests thequantity on hand for a given com-modity. The agent would thendetermine the requested quan-tity and send a response to therequester. The Inventory agentcould also be notified of the eventthat some of the inventory hasbeen removed and is being sentto a customer. The agent processwould then recompute the quan-tity on hand and determinewhether a reorder point has beenreached. If so, the Inventory agentwould send out requests for bid tothe appropriate vendors.

Agent Processing Overview

For an agent to interact with itsenvironment, it must be ableto detect and effect its environ-ment but also to understandwhat it is detecting, to select anappropriate response, and to acton it. In other words, for the agentto be an interactive entity, italso requires some degree ofautonomy and adaptivity. The



percept

sensors

action

effectors

agentenvironment

Figure 13 — Agents interact with their environments.

Output

(initiated actions)

Input

(percepts)

Process

Agent

Environment

and other agents

Figure 14 — An agent and its environment.

internal structure of an agent isshown in Figure 15.

State

Just like objects, agents can havea form of memory, referred tohere simply as state. Stateincludes data that it needs tooperate effectively, which theagent maintains or can access insome manner (such as via a data-base or automated blackboard).For example, an Inventory agentwould need to maintain informa-tion, such as the reorder point fora particular product or the algo-rithm needed to compute thereorder point. The agent mightalso maintain a list of valid prod-uct vendors that it can contact forreordering — or at least knowwhere the information might befound. The agent can even makeinferences about its state or thestate of other entities in its envi-ronment. State, then, can bemetaphorically interpreted asanything that the agent can“know” or “believe” about itselfor its environment.

Detectors

Detectors provide the interface tothe agent’s environment. Theyreceive input that may interestthe agent. Detectors can bothreceive input sent to them oractively scan the environmentfor it. For the more passive soft-ware agents, this could includereceiving messages and eventnotifications. Active agents mightscan message boards or monitorevent occurrences. In particular,

event-monitoring agents (in EDAs)are specialized to recognizeparticular kinds of state changesin the environment. For example, database event agents couldwatch for certain kinds of changesin a database such as metamodelmodifications, customer addresschanges, or the database goingoffline. If any of these events aredetected, the event monitor couldnotify the appropriate agents thatcan react appropriately to theevent. In EDAs, this is particularlyuseful. Robotic agents mayemploy such mechanisms ascameras, global positioning hard-ware, or infrared devices fordetectors.

Parse Input

Once the detector receives input,the agent needs to analyze andunderstand the input. This actiontypically involves recognizing theinput, normalizing it, and dealingwith any inconsistencies in thecontent. For conventional OOsoftware, there is virtually no pars-ing. In the OO “signature,” thefirst parameter identifies the

object, the second specifies therequested operations, and theremaining input contains the vari-ables required for the operationexecution. Because the signaturesyntax is already defined in thismanner, the message does notneed to go through the DetermineAction step. Instead, it goes imme-diately to invoke its specifiedmethod.

In a system with a more complexstyle of input, the OO approachcannot provide a satisfactorysolution. For example, someagents need to process complexsyntaxes and receive their inputin one or more languages, suchas SQL, Prolog, BPEL4WS, BPML,or ebXML. Furthermore, theymay need to understand ontolo-gies expressed in RDF or OWLbefore any further action can bedetermined. For event-monitoringagents, this parsing can meanlooking for patterns in the input.Event pattern recognition is impor-tant for agents, especially forthose that participate in mecha-nisms for self-healing systems.



Input OutputEff

ecto

rs

De

tecto

rs

Parse input

Determine action

Invoke action

AgentState

recognize

deal with inconsistencies

normalize representation

interpret available data

examine goals

form plan

determine mechanism

invoke chosen actions

Figure 15 — The basic internal architecture of an agent.

Granted, a parsing activityrequires greater complexity thanthe traditional object. In fact, notall agents will require sophisti-cated parsers. Yet, when systemrequirements demand richerbehavioral control at executiontime, agents will be able toaccomplish what traditionalobjects do not. Parsing extendsthe notion of the semantic net toenabling the semantic approachto system interaction, in general.

Determine Action

As suggested earlier, traditionalOO recommends that the methodrequested of an object be deter-mined before a message is sent tothe object, because each objectclass defines separate methods,one for each kind of message“signature.” If the number of pos-sible methods is tangible, this isnot a problem; for those situationsrequiring hundreds or thousandsof possibilities, such an approachis impractical. For example, insur-ance companies have hundreds ofways of rating a policy; retailersand shipping companies have anequally large number of rules andprocedures for pricing a productor service. To support this need inan agile and adaptive manner,the agent approach also hasthe option of determining theappropriate method after itreceives a request for service. Inthis way, a policy agent can selectthe appropriate method to rate apolicy based on the requester'sneeds. Order agents can select

the appropriate pricing algorithmwhen the customer submits order.

Another function within theDetermine Action step could beto determine the appropriategoal for further consideration.For example, an Inventory agentcould detect each time a productwas pulled from a warehouse bin.For each instance, the agent mightneed first to decide whether theproduct should be reordered or,if it is moving too slowly, discon-tinued altogether. Each of thesegoals requires a different process.One might involve sending outand processing bids from otheragents to refill the bin; the othermight notify customer agents thatthe product will no longer beavailable.

The Determine Action processmight involve a high degree ofcomplex process in its own right.Can this be accomplished usingan OO approach? Absolutely.However, some agent platformshave mechanisms that supportaction determination directly —instead of having the programmerdo extra work in design, construc-tion, testing, and maintenance.Agents, by nature, are adaptive.

Invoke Action

Once selected, the appropriateaction needs to be invoked. Theaction execution may involveother actions within the agent, inwhich case, messages can be sentdirectly to the agent’s own detec-tors. If the action involves otheragents, the messages need to be

sent to them. The agent controlsits autonomy by using its ownresources or having them pro-vided by other agents. It deter-mines what it can do and what itmust delegate. In this sense, theagent manages itself — ratherthan just being managed.

In addition, some agents candetermine when their internalactions are not functioning asexpected. The agent can alsocommunicate to itself that the sit-uation needs to be remedied —by sending itself a message andletting the Determine Action stepdetermine which action should betaken next. In other words, theagent can have its own processingengine. For BPM systems, thismechanism can then be handledin a distributed manner usingagents, rather than by centralizedcontrollers that will not scaleunder large processing volumes.

Effectors

When messages need to besent or events posted outside theagent, effectors are the connec-tion with the agent’s environment.It is the mechanism an agentuses to interact with and mobilizewithin the environment. Thiscould involve sending messagesthough a message bus or activat-ing a hardware device such as aprinter or robot arm.

Agents Can Be Multiprocessing

Agents can be single threadedor designed to process inputconcurrently (see Figure 16).



Agents that can support concur-rency might also have a processcontroller that coordinates thevarious internal actions in somemanner.

Reasoning, Planning, andScheduling

A common approach for agentscan involve reasoning, planning,and scheduling (see Figure 17). Inthis approach, an agent can deter-mine the appropriate action byhaving a reasoner step considerwhat it knows about its stateand the state of its environment(including that of the servicerequester). Additionally, thereasoner agent can identify theappropriate goals for acting basedon the detected input and stateknowledge. Here, the agent mayoptionally weigh the importanceof certain goals (when there areseveral goals to choose from orconflicting goals). For example,a scheduling agent may have agoal to ensure that all the tasksin its care are executed on time.Another goal might be that cer-tain priorities must be given toimportant tasks. The two goals,however, may conflict whenpriority tasks cause the delay ofother tasks. Here, the agent mayhave to weigh the cost of bothand determine which service itintends to perform under a givencircumstance. A planner thendetermines the actual stepsneeded to execute the agent’s

intension. For example, thescheduling agent may need toensure a particular product iscreated for given order. The agentmay need a make-or-buy decisionresulting in a process that eitherfabricates the product inhouse orpurchases a fabricated item froman external source. Some agentplanners dynamically assemblethe planned set of steps to fit eachoccasion; others have a library

of prefabricated processes fromwhich to choose.

The planned process is then sentto the agent’s scheduler to deter-mine which resources will beassigned and when the processwill be scheduled.5

On top of this, an agent mayalso employ a coordinatorthat ensures that the agent’s



Agent

•••

State

Input OutputEff

ecto

r

De

tecto

r

Parse input

Determine action

Invoke action

Invoke action

Invoke action

Determine action

Determine action

Parse input

Parse input

Figure 16 — Concurrent processing within the agent is possible.

•••

•••

Eff

ecto

r

De

tecto

r

Parse input

Determine action

Invoke action

Parse input

Determine action

Invoke action

Coordinator

Reasoner Planner Scheduler

SchedulePlan

Capabilities

Intentions

Desires, goals

Beliefs

Input Output

State

Agent

Figure 17 — Determining the appropriate action can involve reasoning, planning, and scheduling.

5Agents using this approach are sometimes referred to as “BDI” agents because they employ beliefs, desires, and intentions. Beliefs refer to whatthe agent “believes” the state of its environment and other agents to be; desires basically refer to the goals the agent could attempt to achieve; andintentions refer to the set of goals the agent actually intends to achieve under a given circumstances.

concurrent processing works har-moniously: managing conflictsand optimizing the overall func-tioning of the agent.

But Keep In Mind …

Agent-based systems do notrequire complex forms of agentslike those previously described.For example, ant colonies havebeen very successful for sevenmillion years. An estimated 10%of the Earth’s animal biomass iscomposed of ants, which vastlyexceeds the human population.An ant has about 250,000 braincells; humans have about 10 bil-lion. Ants can lift 20 times theirown body weight. (They maybe small, but they’re efficient!)

What if it were as hard to get yourdeliveries off schedule, as it is tokeep ants out of your kitchen?(See Figure 18.) In other words,agent systems do not requiresophisticated agents to yieldimpressive results. With simplerules and high fault tolerance,the colony thrives — a great first-system model.

Agents and Objects

Just how different — or similar —are agents and objects? Somedevelopers consider agents to beobjects, except with more bellsand whistles. Others see agentsand objects as different eventhough they have many thingsin common. However, both

approaches envision using objectsand agents together in the devel-opment of software systems. Inother words, objects and agentsare two distinct notions — eachhaving its particular place in soft-ware development. The importantpoint here is that the agent-basedway of thinking brings a usefuland important perspective for sys-tem development, which is differ-ent from (while also similar to)the OO way.

Evolution of ProgrammingApproaches

Figure 19 illustrates one wayof thinking about the evolutionof programming languages.Originally, the basic unit of soft-ware was the complete programwhere the programmer had fullcontrol. The program’s state wasthe responsibility of the program-mer and its invocation determinedby the system operator. The term“modular” did not apply becausethe behavior could not be invokedas a reusable unit in a variety ofcircumstances.

As programs became more com-plex and memory space grew,programmers needed to introducesome degree of organization totheir code. Modular programmingemployed smaller units of codethat could be reused under a vari-ety of situations. Structured loopsand subroutines were designedto have a high degree of localintegrity. While each subroutine’scode was encapsulated, its statewas determined by externally sup-plied arguments, and it gained



What if it were as hard to get

deliveries off schedule ...

... as it is to keep ants

out of your kitchen?

Figure 18 — Ant-like simplicity is still very successful. (Source: [7].

Nonmodular Modular ModularModular

InternalInternalExternalExternal

ExternalExternal

(CALLed)

External

(message)

Internal

(rules, goals)

Monolithic

Programming

Agent

Programming

Object-Oriented

Programming

Modular

Programming

Unit

Behavior

Unit

State

Unit

Invocation

Figure 19 — Evolution of programming approaches (Source: [7].)

control only when invoked exter-nally by a CALL statement. Thiswas the era when the primaryprogramming unit was based onmodularizing procedures.

In contrast, object orientationadded to the modular approachby maintaining its segments ofcode (or methods) as well asby gaining local control over thevariables manipulated by its meth-ods. However, in traditional OO,objects were considered passivebecause their methods wereinvoked only when some externalentity sent them a message.

Agents and Objects Together

Software agents have their ownthread of control, localizing notonly code and state but their invo-cation as well. Such agents canalso have individual rules andgoals, making them appear like“active objects with initiative.”

In other words, the agent candetermine when and how itacts. An agent-based approachis employed when a particularsituation requires that process-ing be decentralized and self-organized, instead of centrallyorganized. Any centrally organizedprogram written to handle the antsimulation (refer back to Figure 1)would have been far too cumber-some. It would have required asingle set of top-level rules tellingeach ant precisely what to do inevery conceivable situation. Notonly would such an applicationbe touchy and fragile, it wouldlikely end up looking jerky and

unnatural — more like an ani-mated cartoon than animated life[9].

Yet most developers tend to buildcentrally organized applications.They are also biased toward OOnotions, such as class, association,and message. While these con-structs are useful for a certaincategory of applications, they donot directly address the require-ments of agents. As we have seen,agents have such characteristicsas autonomy, mobility, andadaptability.

Furthermore, business users liketo express other concepts, suchas rules, constraints, and goalsand objectives as well as rolesand responsibilities. In short, theagent-oriented approach distin-guishes between autonomous,interactive, mobile entities(agents) and the passive onesof conventional OO (objects).This does not mean that objectorientation is dead or passé. Awell-designed, agent-based sys-tem uses both objects and agents— just as real-life organizationsemploy a balance of bothactive and passive elements.Furthermore, object technologycan be used to enable, rather thandrive, agent-oriented technology.

CONCLUSION

Making Things Possible withAgents — Not Just Faster, Better,and Cheaper

Using agents has producedapplications faster and cheaperthan other approaches do. For

example, DHL found that its pric-ing and tracking applicationscould be developed 50%-80%faster using agent developmenttools like those available fromOSLO or Agentis Software (www.agentissoftware.com). Time tomarket was reduced and ROIincreased. This does not meanthat agent development toolswill always reduce functionpoints, but it does for someclasses of application.

The most compelling case forusing agent technology is whenagents can enable solutions thatwere either impossible or verydifficult using conventionalmechanisms. At manufacturingcompanies like Deere & Company,several hours each night are typi-cally dedicated to computing opti-mized production schedules forthe shop floor’s use the next day.When an organization has thetime to use techniques like these,agent technology is probablyunnecessary. However, what ifJohn Deere’s IT department deliv-ered its shop-floor schedule at thebeginning of a day when severalkey employees were out sick ora vital piece of machinery in theassembly line broke down? If theyneeded to stop the assembly linefor several hours to recreate theproduction schedule, this solutionclearly is not the answer. Underthese circumstances, an agent-based approach should be consid-ered. Instead of starting over toreoptimize the schedule for all theresources, agents can be used toproduce a local optimum. If one



resource is no longer available,agents can look for a replacementand “contract” with alternativeresources. If the resources aregoing to be delayed, agents ina supply chain can determinewhich way to solve the delaywith the least impact — usingautomated speeds.

Even techniques such as linearprogramming, neural networks,and genetic algorithms are some-times used for similar situations.Again, when these provideaccurate results in a timely mat-ter, agents are not necessary.However, for large complex sys-tems, such approaches can some-times become costly to configureor reconfigure and also involvescalability problems. In such situa-tions, they are often run offlineand do not respond rapidly todynamic changes in environment.When this is the case, the horizon-tal approach of agent technologycan significantly outperform acentralized scheduling approach.

Companies like IAI have devel-oped systems involving over100,000 agents that can handlethe resource rescheduling in amanner of seconds — insteadof hours. General-purpose agentplatforms, such as the OSLO Suite,can be used, or a more special-ized package can be chosen. Forexample, agent-based logisticssoftware is available fromCougaar and Whitestein, andevent-driven sense-and-respondsolutions from Rhysome. Manyother large software companies

already employ agents inside theirofferings but do not refer to themby name. Agent technology makesit possible to do things that werepreviously either impossible orcould not be accomplished in anacceptable time frame.

Where To Go From Here

If you think agent technologymight be appropriate for yourorganization, start by readingsome books on agents (such as[10]). Then, talk to companiesthat provide or use agent technol-ogy (such as those mentioned inthis report). In doing so, you willget an idea of how and where touse agents. Remember, though,that the biggest breakthroughwith agents is that they are anevolution of existing technologies.What is revolutionary is the waywe think about and use agents todesign IT systems.

Going Forward to the GlobalIT Organization

Agent technology is now neces-sary to reduce costs; to improveefficiency and effectiveness; andto support the requirements ofindividuals, groups, companies,and universities as they collabo-rate globally. More importantly,it will enable us to create andsupport a whole class of IT appli-cations and approaches thatwe previously could not havedeveloped.

As Pulitzer Prize–winning authorand columnist Thomas Friedmansaid so eloquently:

[We need] a global, Web-enabled platform for multipleforms of collaboration. Thisplatform enables individuals,groups, companies, and uni-versities anywhere in theworld to collaborate — forthe purposes of innovation,production, education,research, entertainment,and, alas, war-making — likeno creative platform before.This platform now operateswithout regard to geography,distance, time, and, in thenear future, even language.Going forward, this platformis going to be at the center ofeverything. [5]

This emergence of new businessprocesses is resulting in new ITpractices and approaches —where the two mutually reinforceeach other. Now that individuals,groups, companies, and universi-ties are interacting on a globalscale, IT must not forget its sup-porting role. Agents are a keyingredient in IT globalizing itself.

REFERENCES

1. Baker, Albert D., H. Van DykeParunak, and Kutluhan Erol.“Agents and The Internet:Infrastructure for MassCustomization.” IEEE InternetComputing, Vol. 3, No. 5,September 1999, pp. 62-69.

2. Bonabeau, Eric, Marco Dorigo,and Guy Theraulaz. SwarmIntelligence: From Natural toArtificial Systems. OxfordUniversity Press, 1999.



3. British BroadcastingCorporation (BBC) News.“Humble Ant Is a SoftwareSaviour.” 28 January 1998(http://news.bbc.co.uk/2/hi/science/nature/51337.stm).

4. Ernst & Young Center forBusiness Innovation. “EmbracingComplexity: Exploring theApplication of Complex AdaptiveSystems to Business.” ChristopherMeyer (ed.). Proceedings from the1996 Colloquium on the BusinessApplication of Complexity Science,Boston, Massachusetts, USA, 17-19July 1996.

5. Friedman, Thomas L. TheWorld Is Flat: A Brief Historyof the Twenty-First Century.Farrar, Straus and Giroux, 2005.

6. Luck, Michael, Peter McBurney,Onn Shehory, and StevenWillmott. “Agent TechnologyRoadmap: Computing asInteraction (A Roadmap forAgent Based Computing).”AgentLink, 2005.

7. Parunak, H. Van Dyke.“DESK: Density-Based EmergentScheduling Kernel.” Slide presen-tation to DTE Energy, materialadapted in collaboration withNewVectors, 2000.

8. Resnick, Mitchel. Turtles,Termites, and Traffic Jams:Explorations in Massively ParallelMicroworlds. The MIT Press, 1997.

9. Schoonderwoerd, Ruud, OwenHolland, Janet Bruten, and LeonRothkrantz. “Ant-Based LoadBalancing in TelecommunicationsNetworks.” Adaptive Behavior,Vol. 5, No. 2, 1996, pp. 169-207.

10. Singh, Munindar P,. andMichael N. Huhns. Service-Oriented Computing: Semantics,Processes, Agents. Wiley, 2005.

11. W3C. “Web ServicesArchitecture.” W3C WorkingGroup Note, 11 February 2004(www.w3.org/TR/2004/NOTE-ws-arch-20040211/)

ABOUT THE AUTHOR

James Odell is a Senior Consultantwith Cutter Consortium’sEnterprise Architecture practiceand a regular contributor to theEnterprise Architecture advisoryservice. He is a consultant, writer,and educator in the areas of agent-based and object-oriented (OO)systems. Throughout most of his35-year career, Mr. Odell has beenheavily involved in developing bet-ter methods to understand, com-municate, and manage systemrequirements. He was one of theearly innovators of informationengineering methodologies.

Mr. Odell is one of the first practi-cal implementers of OO analysisand design. Working with theOMG and other major method-ologists, he continues to innovate

and improve OO methods andtechniques. He participated in thedevelopment of UML, and is thecochair of the OMG’s Analysis andDesign Task Force as well as theAgents Special Interest Group.

Mr. Odell conducts internationalseminars and workshops andprovides consulting to majorcompanies worldwide. Formerly,Mr. Odell was the principal con-sultant for KnowledgeWare, Inc.,where he pioneered and taughtthe concepts of data modeling,information strategy planning,and CASE technology application.He now works as an independent

consultant to advance the

state of agent technologywithin the OMG and the com-puter industry.

Mr. Odell has coauthored, withJames Martin, several books includ-ing Object-Oriented Analysis andDesign, Object-Oriented Methods:Pragmatic Considerations, andmost recently, Object-OrientedMethods: A Foundation, UMLEdition. He also has publishedthree books on agent-orientedsoftware engineering.

Mr. Odell’s clients includeAmazon.com, Netscape, Chrysler,Capgemini, DHL, NASA, and othermajor companies across variousbusiness sectors in 19 differentcountries. He can be reached [email protected].



agent technology

Documents