Performance Comparison of Data Exchange in Agent-BasedCollaborative Design Process

Wei-Jung Shiang, Hsin Rau, and Yu-Hsin Lin

Abstract-Multi-agent systems have been proposed tosupport the destributed environment of collaborative design formany years. This research proposed two differentcommunication architectures based on two major distributedobject models, Java RMI and CORBA/Java. These twoarchitectures could be applied in any agent systems supportingJava technology. Experiments were designed and conducted tomeasure the performance of data exchange between twoheterogeneous agent systems with these two architectures. Inthe experiments, distributed object model, number of clients,data size and buffer size were defined as independent variables.Communication time was defined as dependent variable. Basedon the experimental result, all independent variables aresignificant, and the performance of RMI is better intransferring files. Therefore, the communication architecturebased on RMI is recommended for file exchange between JADEand AgentBuilder systems in practical applications.

I. INTRODUCTION

A DAPTING to a dynamically changing world,business organization focuses on its core competencies

and collaborates with other companies to develop newproducts. Collaborative design is regarded as joint problemsolving with a shared goal of developing successful products[9]. Group members work with others from differentcompanies, and attempt to fmd design solutions that aresatisfying to all stakeholders. Computer-supportedcollaborative design should take care of the issue of memberinteractions in the design process, such as negotiation andevaluation activities.

Heterogeneity of the team members, professions, andinformation systems involving in the collaborative designneeds to be well managed during the design project.Software agent with autonomy has been treated as theappropriate solution to manage the complexity of thecollaborative design projects. Several multi-agentarchitectural frameworks have been proposed or developedto support the collaborative design process [5, 6]. In mostresearch papers, the problem of applying heterogeneousagent systems in the design process is not well addressed.

Manuscript received March 1, 2009. This work was supported in part bythe National Science Council of Republic of China under the grant NSC97-2221-E-033-042.

Wei-lung Shiang and Hsin Rau are with the Industrial and SystemsEngineering Department, Chung Yuan Christian University, 200 Chung PeiRoad, Chung-Li, 32023 Taiwan, (phone: 886-3-265-4418; e-mail: wjs001@cycu.edu.tw).

Yu-Hsin Lin is with the Department of Industrial Engineering andManagement, Ming-Hsin University of Science & Technology, 1 HsinhsinRd., Hsinfong, Hsin-Chu, 304 Taiwan.

978-1-4244-3541-8/09/$25.00 ©2009 IEEE

However, it seems not impossible that different companiesmay use various agent systems in the real world. In such acase, agent communicating with other agents in variousagent systems is an important issue for the development ofefficient and robust applications.

Agent communication issue involves the defmition of acoordination model and associated coordination architectureor a related coordination language. Four coordinationmodels have been described in [3], and agents in directcoordination model usually coordinate with RPC (remoteprocedure call)-like primitives or message passing. Agentcommunication should not only limit to the message passingbetween agents in collaborative design process. At least,circulating of CAD files is common in the designenvironment.

In Java, there are two suitable distributed object models,Common Object Request Broker Architecture (CORBA) andRemote Method Invocation (RMI) [8]. These two modelscan be applied to support agent communications in Javasupported heterogeneous agent systems. In this paper, twoagent communication architectures based on two distributedobject models, Java RMI and CORBA/Java are proposed fortwo popular multi-agent systems. Performance is regarded asthe most important criterion in choosing the betterdistributed object model. Several experiments are designedto measure the performance of these two models, and theresults are analyzed to fmd major factors in transferring filesbetween agents. With the analysis of CORBA/Java and JavaRMI, this paper presents the suggestion of choosingdistributed object model.

II. LITERATURE REVIEW

The requirements of a multi-agent system for supportingcollaborative design were summarized in [5]. They arguedthat the agent system should efficiently manage the designinformation in the process and build up an effectivecommunication environment to help multidisciplinary designteam. They also pointed out that multi-agent systems andsoftware agents should be effective techniques for providingsupport for the complex tasks in collaborative design.However, there is no discussion about heterogeneous agentsystems in the distributed design environment.

In multi-agent systems, several software agentscooperatively conducting a design task need to communicateto each other and to synchronize their activities. An agentcoordination model is a framework, which covers the issuesof creation and deletion of agents, communications among

434

Fig. I. The Java RMI communication architecture.

III. ARCHITECTURES AND EXPERIMENTS

This paper emphasizes the agent communication issue inmulti-agent systems for collaborative design process. Inorder to address this issue, agent communication architecturefor each distributed object model should be proposed, andthe performances of communication architectures should becompared to establish reference information for agentsystem developers. Two agent direct communicationarchitectures based on two distributed object models, JavaRMI and CORBA/Java, are proposed and shown in Fig. 1and 2 for two heterogeneous multi-agent systems, JADE andAgentBuilder. Communication time is regarded as the mostimportant criterion in deciding the better distributed objectmodel in these two architectures.

FIPA (Foundation for Intelligent Physical Agent)specifications for interoperable intelligent multi-agentsystems [2]. JADE includes agent management system ,directory facilitator, agent communication channel, andgraphic tools . A JADE platform is a distributed environmentconsisting of run time containers. In JADE, each agent has asort of mailbox where the JADE runtime posts messagessent by other agents . Whenever a message is posted in themessage queue the receiving agent is notified. Forinter-container message passing, RMI is applied fordistributed object computing. For agents live on the samecontainer, an event object representing the message is clonedand passed to the receiver.

AgentBuilder is an integrated software package fordeveloping software agents and agent-based applications [1].It consists of two major components: the toolkit and therun-time system. The toolkit provides graphical tools forsupporting the agent construction process, designing anddeveloping networks of communicating agents, and defmingbehaviors of individual agents. The run-time system consistsof the agent program and an agent engine. The agent engineprovides an environment for execution of agent programs.All components of AgentBuilder are coded in Java, andagents created with the AgentBuilder toolkit can be executedon any Java virtual machine. In such a case, AgentBuilder isselected as another platform to develop collaborative designprocess in this paper, and experiments are designed to testthe communication performance between Jade andAgentBuilder agents with Java RMI and CORBA/Javamodels.

JADEenvironmentAgcntlluildcr environment

agents, and the synchronization and distribution of agents [7].Identifying the coordination requirements of the agents forcollaborative design and fulfilling those needs with anappropriate coordination model are fundamental tasks fordesigning a multi-agent application.

Based on the combinations of spatial and temporalcoupling, four coordination models have been described asdirect, meeting-oriented, blackboard, and tuple-basedcoordination models [3]. In direct coordination models,agents usually coordinate using RPC-like primitives ormessage passing. The majority of the Java-based mobileagent systems rely on direct coordination model such asAglets , Voyager, Ajanta and Grasshoppers. In this paper, theperformance comparison focuses on the direct coordinationmodel applied to support data exchange betweenheterogeneous agent systems.

The CORBA from the Object Management Group (OMG)defmes a language-independent framework for writingdistributed, object-oriented applications [10]. Thesedistributed objects can interoperate with each other. Beforean object can invoke a method on another object, distributedobjects must be identified by object references, which aretyped by IDL interfaces. The Object Request Broker (ORB)is the distributed service that locates the remote object on thenetwork, communicates the request to the remote object,waits for the results and communicates those results back tothe client. Several vendors provide CORBA products forJava and Java 2 ORB is used in this paper to developexperimental applications.

The Java RMI provides for remote communicationbetween objects coded in the Java programming language[11]. RMI system allows an object running in one Javavirtual machine to invoke methods on a remote objectrunning in another Java virtual machine. There are twoseparate objects in RMI applications, a server and a client.The functions of a server program are creating remoteobjects, making references to these objects , and waiting forclients to invoke methods on these remote objects. A clientprogram is for obtaining a remote reference to any remoteobjects on a server and then invoking methods on them.

To measure the performance of agent systems in dataexchange, distributed object model should be included as afactor for more realistic implementation. Java RMI andCORBA/Java are suitable for distributed objectcommunication, and they can be applied in heterogeneousagent systems for agent communication. There are manydecision criteria for choosing the distributed object model,such as features, performance, maturity, etc. [8]. Cabri et al.[3] argued performance of the agent application is the majorfactor for deciding the suitable coordination model. Aquantitative analysis with measuring the performance ofdifferent application designs was suggested for identifyingthe appropriate approach.

In this paper, JADE (Java Agent DevelopmentFramework) is selected as one of the platform to developcollaborative design process. JADE is compliant with the

435

In these two architectures, JADE system acts as the serverside and AgentBuilder is for the client side. In Fig. 1, theagent generated in JADE system binds service in RMIregistry server. The agent generated in AgentBuilder looksup the remote JADE agent by its name in the RMI registryserver, and then invokes a method on the remote agent. InFig. 2, JADE and AgentBuilder agents communicate andpass information back and forth in CORBA model, and theprocess is similar to the RMI model except some underlyingmechanisms. The comparisons of these two architectures aresummarized in Table I.

TABLEICOMPARISONS OF TWO DIRECT COMMUNICAnON ARCHITECTURE

RMI architecture CORBA architecture

Interface

Object Reference

Bus

Class file

Registry server

Java socket

IDL file

Name server

ORB

final public void setFile(String y) throwsjava.rmi.RemoteException{ try

{filelnputStream= new FilelnputStream(newFile(y));}

catch(IOException e){ e.printStackTrace(); } }

final public byte[] getFile() throwsjava.rmi.RemoteException{ try {

while (true) { length =filelnputStream.read(buffer);

if(filelnputStream.available() <l)setcont(false);return buffer; }

{... }protected void setup()

{addBehaviour (new OneShotBehaviour (this)public void action() { try {Serverlmpl server = new Serverlmpl();

UnicastRemoteObject.exportObject(server);

Ageniltilder agentscode

Fig. 2. The CORBNJava communication architecture.

In order to measure the performance of these twoarchitectures, experiments were designed to test thecommunication time of transferring files between agents intwo multi-agent systems. In this RMI application, thecompute engine is a remote object on the JADE server, andthe server programs consist of an interface and a class. Theinterface defines method names that can be invoked from theAgentBuilder client, and the class program provides theimplementation of the methods for transferring files. Themajor part of a RMI server class program in JADE system islisted as following:

import java.rmi.*;import jade.core.Agent;import jade.core.Runtime;import jade.core.behaviours.*;import java.rmi.Naming;import java.rmi.server.UnicastRemoteObject;

public class Serverlmpl extends Agent implementsServer,

Serializable{ArrayList dllist=new ArrayList();

FilelnputStream filelnputStream;final public void setBuffer(int x) throwsjava.rmi.RemoteException{

buffer=new byte[x];length=x;... }

Naming. rebind (" / /_/NetServer", server);_ } catch (Exception e) {_}

The same application in CORBA/Java was also coded andthe major part of the program is listed as following. Thereare some unavoidable differences in the initial invocations tothe ORB and other details.

import org.omg.CORBA.*;import org.omg.CosNaming.*;importorg.omg.CosNaming.NamingContextPackage.*;import java.rmi.*;import jade.core.Agent;import jade.core.behaviours.*;

public class JADECORBA extends Agent{protected void setup() {

addBehaviour(new OneShotBehaviour(this)private boolean finished=false;String [] args;

public void action() { tryProperties props = new Properties();props.put("org.omg.CORBA.ORBlnitialPort",

"2156") ;ORB orb = ORB.init(args, props);Serverlmpl myserverl = new Serverlmpl();

orb.connect(myserverl);

org.omg.CORBA.Object oReforb. resolve_initial references("NameSer

vice");NamingContext nContext =

NamingContextHelper.narrow(oRef);NameComponent compl =

new NameComponent("Serverl","2156");

NameComponent pathl[] = {compl};

nContext.rebind(pathl, myserverl);

java.lang.Object sync = newjava.lang.Object();

436

Independent LevelsFactors

IV. DATA ANALYSIS

In this paper, an experimental design with fourindependent factors was investigated to study the effect ofdirect communication architecture in inter-agent tasks oftransferring files. There were total 360 experimental data

Where ~ is the overall mean effect , a, is the effect of theith level of object model, ~j is the effect of the jth level ofclient number, Yk is the effect of the kth level of file size, PI isthe effect of the lth level of buffer size, a~ij, aYik, and aRi/are the effects of interactions between object model andclient number, file size and buffer size, and E is the randomerror. For each combination, five replication data werecollected to increase accuracy of this study. The analysis ofvariance (ANOVA) was applied to test the hypotheses of noeffect for all independent factors on communication time.The results can be used as references to select the betterdirect communication architecture for multi-agent systems incollaborative design process.

TABLE IIFACTORIAL DESIGN OF THE EXPERlMENT

0.0000.0000.0000.0000.0000.0000.000

P valueF2042 .03341.022367 .21262 .6184.25

372.557.08

1000K

I232232

l OOK

Bufh 2 (byte)

328359

TABLE IIIANOVA TABLE

10K

Degree of FreedomSource

ErrorTotal

Object Model (A)Client Number (B)

File Size (C)Buffer Size (D)

A xBA xCA xD

Furthermore, the interactions between object model andother independent factors also significantly affect thecommunication time with the same significance level. Itmeans that RMI might have different effect on other factorsthan CORBA in transferring files. In Fig. 4, even large filesmay increase the transferring time, CORBA with larger filesizes might have larger effects in increasing transferringtime than RMI with larger file sizes. Based on theexperimental results, RMI architecture with lOOK bytesbuffer size had the best performance in this study. However,if the Java RMI or CORBA/Java technologies upgrade, theperformance may change . The technical reason why the RMIarchitecture outperformed the CORBA one is not included inthis experiment and this issue is under further research . As amatter of fact, the better fit between object model and otherfactors is essential in developing an effective multi-agentcollaborative design process.

6000 ,------ - - - - - - ------,

!5000 ~~ 4000

'*J! 3000

~ 2000

:< 1000

Datameansfor transferring tme

Fig . 3. The main effect plot of buffer size

collected in this study. The ANOVA was calculated withSPSS statistic software, and the result was summarized inTable III. Based on the significance level of 0.05, it isconcluded that main effects of all independent factorssignificantly affect the file transferring time between agents.It is straightforward to interpret that larger file size andclient number may take longer time to finish the task. Fromthe main effect analysis , it also showed that RMIarchitecture might spend less time than CORBA intransferring files. However, in Fig.3, it showed that buffersize with lOOK bytes might have better performance in thisexperiment. The reason might be that smaller buffer sizewould increase more computer time in writing files into thebuffer area, and larger buffer size would increase thenetwork transferring time. Therefore, for the file sizebetween one to 20 Mega bytes, the buffer size with lOOKbytes is the best choice.

(1)

JavaRMICORBAIJava

I23I51020

10K

lOOK1000K

I2I23I234I

23

File Size(MB)

Object Model

Client Number

Buffer Size (byte)

synchronized (sync) { sync.wait(); }}catch(Exception e) {_IThe Java source codes were compiled and executed with

JDK 1.4 on Windows XP personal computers used as JADEserver side and AgentBuilder client side. In order to simulatethe real word environment, computers were distributed twokilometers apart and connected through ADSL networks.Buffer was used on the server side to improve the filetransferring efficiency and three different buffer sizes weretested in the experiment. This experiment also testedmultiple clients invoking methods on a single server objectand there were up to three clients. Considering dataexchanges between collaborative designers were most CADfiles, files in four different sizes were transferred to measurethe communication times for this study.

There were four independent factors in this experimentand their levels were listed in Table II. Communication timewas measured and regarded as dependent variable. Theperformance observations for this experiment were definedas a linear statistical model

Y =Jl + a, + fJj + Yk + PI

+ (afJ ) ij + (ay) ik + (ap ) if + eijkl

437

Datameans for transferring time

14000

120008:, 10000

i 8000 I-+- IRMI

~ 6000 ~2 CORBA

~ 4000

~:a2000

0

1 5 10 20

File size(M8)

Fig. 4. The interaction plot of file size and object model

V. CONCLUSIONS

This paper emphasized the agent communication issue inheterogeneous multi-agent systems for collaborative designprocess. Two agent direct communication architecturesbased on two distributed object models, Java RMI andCORBA/Java, were proposed and tested for two multi-agentsystems, JADE and AgentBuilder. Based on theexperimental result , distributed object model , number ofclients, data size and buffer size were significant intransferring files between heterogeneous agents. Ifcommunication time was considered as the majorperformance index, RMI outperformed CORBA/Java intransferring files between agents . For future research, moredistributed object models or other communicationtechnologies should be compared on more variousmulti-agent systems.

REFERENCES

[I] Acronymics, Inc. Available : http://www.agentbuilder.com.[2] F. Bellifemine, A. Poggi, and G. Rimassa, "JADE - A

FIPA-Compliant Agent Framework," Proceedings ofPAAM' 99,1999,pp.97-108.

[3] G. Cabri , L. Leonardi , and F. Zambonelli, "Mobile-agent coordinationmodels for Internet applications," IEEE computer , vol. 33, pp. 82-892000.

[4] 1. Cao, X. Feng, 1. Lu, and S. Das, "Mailbox-based scheme for mobileagent communications," IEEE Computer, vol. 35, pp. 54-60, 2002.

[5] C. Chira , O. Chira, and T. Roche, "Multi-agent Support forDistributed Engineering Design." IEAIAIE, LNAI 3533, 155--164(2005)

[6] E.A. Edmonds , L. Candy , R. Jones , and B. Soufi "Support forCollaborative Design : Agents and Emergence," Communications ofthe ACM, 37, pp. 41-47, 1994.

[7] Fortino, G., Russo, W. "Multi-coordination of Mobile Agents: aModel and a Component-based Architecture," Proceedings of theACMSymposium on Applied Computing , pp. 443-450 , 2005.

[8] M. Juric , I. Rozman , and M. Hericko , "Performance Comparison ofCORBA and RMI," Information and Software Technology , vol. 42, pp.915 - 933, 2000 .

[9] T. Kvan, "Collaborative Design : What Is it?" Automation inConstruction, vol. 9, pp. 409-415 , 2000.

[10] Sun Developer Network. Available :http://java.sun.com!developer/onlineTraining/corba/corba.htmI

[II] The Java Tutorial s Available:http://java.sun .com!docslbooks/tutorial /rmi/index.html

438