R. Khosla, N. Ichalkaranje and L. C. Jain

Design of Intelligent Multi-Agent Systems

Studies in Fuzziness and Soft Computing, Volume 162

Editor-in-chief Prof. Janusz Kacprzyk Systems Research Institute Polish Academy of Sciences ul. N ewelska 6 01-447 Warsaw Poland E-mail: kacprzyk@ibspan.waw.pl

Further volumes of this series can be found on our homepage: springeronline.com

Vol. 146. J.A. Gamez, S. Moral, A. Salmeron (Eds.) Advances in Bayesian Networks, 2004 ISBN 3-540-20876-3

Vol. 147. K. Watanabe, M.M.A. Hashem New Algorithms and their Applications to Evolutionary Robots, 2004 ISBN 3-540-20901-8

Vol. 148. C. Martin-Vide, V. Mitrana, G. Piiun (Eds.) Formal Languages and Applications, 2004 ISBN 3-540-20907-7

Vol. 149. J.J. Buckley Fuzzy Statistics, 2004 ISBN 3-540-21084-9

Vol. 150. 1. Bull (Ed.) Applications of Learning Classifier Systems, 2004 ISBN 3-540-21109-8

Vol. 151. T. Kowalczyk, E. Pleszczynska, F. Ruland (Eds.) Grade Models and Methods for Data Analysis, 2004 ISBN 3-540-21120-9

Vol. 152. J. Rajapakse, 1. Wang (Eds.) Neural Information Processing: Research and Development, 2004 ISBN 3-540-21123-3

Vol. 153. J. Fulcher, 1.c. Jain (Eds.) Applied Intelligent Systems, 2004 ISBN 3-540-21153-5

Vol. 154. B. Liu Uncertainty Theory, 2004 ISBN 3-540-21333-3

Vol. 155. G. Resconi , J.1. Jain Intelligent Agents, 2004 ISBN 3-540-22003-8

Vol. 156. R. Tadeusiewicz, M.R . Ogiela Medical Image Understanding Technology, 2004 ISBN 3-540-21985-4

Vol. 157. R.A. Aliev, F. Fazlollahi, R.R. Aliev Soft Computing and its Applications in Business and Economics, 2004 ISBN 3-540-22138-7

Vol. 158. K.K. Dompere Cost-Benefit Analysis and the Theory of Fuzzy Decisions - Identification and Measurement Theory, 2004 ISBN 3-540-22154-9

Vol. 159. E. Damiani, 1.e. Jain, M. Madravia Soft Computing in Software Engineering, 2004 ISBN 3-540-22030-5

Vol. 160. K.K. Dompere Cost-Benefit Analysis and the Theory of Fuzzy Decisions - Fuzzy Value Theory, 2004 ISBN 3-540-22161-1

Vol. 161. N. Nedjah, 1. de Macedo Mourelle (Eds.) Evolvable Machines, 2005 ISBN 3-540-22905-1

R. Khosla N.lchalkaranje L. c. Jain

Design of Intelligent Multi -Agent Systems Human -Centredness, Architectures, Learning and Adaptation

~ Springer

Nikhil Ichalkaranje Professor Lakhmi Jain

Knowledge-Based Intelligent

Engineering Systems Centre

School of Electrical and

Information Engineering

University of South Australia

Adelaide

Mawson Lakes Campus

South Australia SA 5095

Australia

E-mail: Nihil.Ichalkaranje@unisa.edu.au

Lakhmi.Jain@unisa.edu.au

Associate Professor Rajiv Khosla

Business Systems and Knowledge

Modelling Laboratory

School of Business

LaTrobe University

Melbourne, Victoria 3086

Australia

E-mail: R.Khosla@latrobe.edu.au

ISBN 978-3-642-06177-6 ISBN 978-3-540-44516-6 (eBook) DOI 10.1007/978-3-540-44516-6

This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitations, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer-Verlag. Violations are liable to prosecution under the German Copyright Law.

©Springer-Verlag Berlin Heidelberg 2005 Originally published by Springer-Verlag Berlin Heidelberg in 2005 Softcover reprint of the hardcover 1st edition 2005

The use of general descriptive names, registered names trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.

Cover design: E. Kirchner, Springer-Verlag, Heidelberg Printed on acid free paper 62/3020/M - 54 3 2 I 0

Preface

There is a tremendous interest in the design and applications of agents in virtually every area including avionics, business, internet, engineering, health sciences and management. There is no agreed one definition of an agent but we can define an agent as a computer program that autonomously or semi-autonomously acts on behalf of the user.

In the last five years transition of intelligent systems research in general and agent based research in particular from a laboratory environment into the real world has resulted in the emergence of several phenomenon. These trends can be placed in three catego­ries, namely, humanization, architectures and learning and adapta­tion. These phenomena are distinct from the traditional logic­centered approach associated with the agent paradigm. Humaniza­tion of agents can be understood among other aspects, in terms of the semantics quality of design of agents. The need to humanize agents is to allow practitioners and users to make more effective use of this technology. It relates to the semantic quality of the agent design. Further, context-awareness is another aspect which has as­sumed importance in the light of ubiquitous computing and ambi­ent intelligence.

The widespread and varied use of agents on the other hand has cre­ated a need for agent-based software development frameworks and design patterns as well architectures for situated interaction, nego­tiation, e-commerce, e-business and informational retrieval. Fi-

vi Preface

nally, traditional agent designs did not incorporate human-like abilities of learning and adaptation. Researchers today are using soft computing technologies like neural networks to make agents learn and adapt.

This book attempts to focus on the above trends in the design and evolution of the agent technology. In the rest of this introduction we provide a brief outline of the various chapters in this book.

Soft computing agents today are being applied in a range of areas including image processing, engineering, process control, data min­ing, internet and others. In the process of applying soft computing agents to complex real world problems three phenomena have emerged. These include their application in distributed environ­ments, humanization of these agents and optimization of their per­formance. Chapter 1 models these phenomena as part of multi­layered multi-agent architecture. The layered architecture is also motivated by the human-centered approach and criteria outlined in the 1997 NSF workshop on human-centered systems.

Chapter 2 presents the general requirements of the ubiquitous envi­ronment and discusses the use of software agent technology to fa­cilitate service provision and composition for ubiquitous client end systems.

Chapter 3 reports a prototype of the developed agent community implementation in the system "KSNet" and a constraint-based pro­tocol designed for the agents' negotiation. An application of the developed agent community to coalition-based operations support and the protocol are illustrated via case studies of a mobile hospital configuration as a task of health service logistics and automotive supply network configuration.

Chapter 4 proposes architectural styles and design patterns for a multi-agent system which adopts concepts from social theories.

Preface vii

The design parameters included are social, intentional, structural, communication and dynamic. An e-business is included in the chapter.

Chapter 5 presents massive multi-agent systems like unsteady software systems whose behaviour develops some emergent char­acters. These systems are made to learn and to adjust themselves to the situation on environment. Their design is not a usual way of producing all the specifications and a strong validation. They aren't problem solvers. They are systems functioning with a behaviour oriented in the time following their continuous development.

JADE (Java Agent Development Framework) is an "open source" FIP A-compliant software environment to build agent systems. JADE offers an agent middleware to implement efficient FIPA2000 compliant multi-agent systems and supports their development through the availability of a predefined programmable agent model, an ontology development support, and a set of management and testing tools. Chapter 6 describes JADE and its use in three interna­tional projects to develop applications in the fields of: corporate memory management, integration of fixed and mobile networks, and integration of Web services.

Chapter 7 introduces ant algorithms and presents their applications. Chapter 8 presents the development of software architecture for agents to communicate and synchronise with the application in a totally transparent way. The design of multi-agent assistant for fa­cilitating e-commerce activities during web browsing is presented.

Chapter 9 presents a set of techniques for selection, exchange and incorporation of information to help a learning agent to achieve its goals. Chapter 10 presents the criteria related to the adaptation and mutation in multi-agent systems.

viii Preface

Chapter 11 explores the possibility of applying reinforcement learn­ing to acquire new high-level actions for animated learning agents. The chosen algorithm is the deterministic version of Q-Iearning. This allows for easy definition of the task, since only the ultimate goal of the learning agent must be defined. Generated actions can then be used to enrich the animation produced by an animation system. Re­sults achieved when training agents with forward and inverse kine­matics control are also demonstrated and compared.

The final chapter presents the basic concepts related to agent tech­nologies. It includes the implementation of an agent-based architec­ture for information retrieval. It also suggests a new architecture for product search in e-commerce.

We are indebted to the contributors and reviewers for their wonder­ful efforts. Berend Jan van der Zwaag provided excellent contribu­tion in this book. Thanks are due to the Editorial Staff of Springer­Verlag for their assistance.

Nikhillchalkaranje Rajiv Khosla Lakhmi Jain

Contents

Chapter 1. 1 Humanization of soft computing agents Rajiv Khosla, Qiubang Li, and Chris Lai

1 Introduction ............................................................................ 1 2 Human-centered system development framework ..... ... .... ... .. 2 3 Distributed multi-agent architecture ....................................... 4

3.1 Problem solving ontology layer ......................................... 5 3.2 Optimization layer ......... ... ........ ......................................... 9 3.3 Tool or technology layer ................................................. 11

4 Human-centered modelling using soft computing multi-agent architecture ......... ... .......... ..... .... ...... .................. . 14

4.1 Human-centeredness and problem solving agent layer ... 14 4.1.1 CRM model of Internet-banking ................................. 16 4.1.2 Decomposition phase problem solving agent

andCRM .......................... ............ .. ......... ... ....... .... ...... 17 4.1.3 Control phase problem solving agent ......................... 18 4.1.4 Decision phase problem solving agent ....................... 19

4.2 Unstained cell image processing ................. .. ... ... .. ..... ..... 21 4.3 Human-centeredness and technology agent layer ........... 25

5 Conclusion ............................................................................ 26 References ............................................................................ 27

Chapter 2. 31 Software agents for ubiquitous computing Sasu Tarkoma, Mikko Laukkanen, Kimmo Raatikainen

1 Introduction ................. .. .... .. ...... ..... ............. .. .... .... ....... .. .. .... 31 2 Ubiquitous computing .......................................................... 33

2.1 Overview ......................................................................... 33 2.2 Wireless networks and roaming ....... ..... ................... .. ..... 34 2.3 Client devices .................................................................. 35

x Contents

2.4 Location- and context-aware services ........................... .. 36 2.5 Technology support.. ....................................................... 37

2.5.1 Java - the enabling technology for software agents ... 37 2.5.2 Other technologies ...................................................... 38

3 Ubiquitous agents ................................................................. 40 3.1 Overview ......................................................................... 40 3.2 The FIPA architecture .. .... ........................ .. ..................... 44 3.3 Agent platforms ............................................................... 46

3.3.1 JADE-LEAP ............................................................... 47 3.3.2 FIPA-OS and MicroFIPA-OS ..................................... 47

3.4 Proxy-based approaches .................................................. 48 3.5 Agent communication ..................................................... 49 3.6 Events for agents ............................................................. 49

4 Agent-based service provision and deployment.. ................. 50 4.1 Agent and service deployment ........................................ 50 4.2 Service partitioning based on the environment ............... 52 4.3 Example scenario: recommendation service ... .. .............. 54 4.4 CRUMPET ...................................................................... 57

5 Conclusions .......................................................................... 59 References .............................................................. ........ .. ... . 60

Chapter 3. 63 Agents-based knowledge logistics Alexander Smirnov, Mikhail Pashkin, Nikolai Chilov,

and Tatiana Levashova

1 Introduction .......................................................................... 63 2 KSNet-approach: major ideas .............................................. 67 3 Features of agent community in the system "KSNet" .......... 69 4 Communication, interaction and negotiation

in the KL system ..................................... ............................. 74 4.1 Conventional CNP ........................................................... 75 4.2 Constraint-based negotiation ......... .................................. 76 4.3 Modifications of interaction ............................ ....... ......... 77 4.4 Example of utilizing constraint-based CNP .................... 77

Contents xi

5 Implementation of agent community ................................... 84 6 Case study: health service logistics ...................................... 90 7 Case study: virtual supply network ...................................... 93 8 Conclusion ............................................................................ 94

Acknowledgements .............................................................. 96 References ............................... .... ......................................... 96

Chapter 4. 103 Architectural styles and patterns for multi-agent systems Manuel Kolp, T. Tung Do, Stephane Faulkner,

and T.T. Hang Hoang

1 Introduction ........................................................................ 103 2 Organizational architectural styles ..................................... 106

2.1 Applying organizational styles ...................................... 111 2.2 Evaluation ................. .. ...... ... .................... .. .. .............. .. .. 114

3 Social patterns.. ............................................ .. ... ................. 116 3.1 Modeling social patterns ................................................ 117

3.1.1 Social dimension .. .. ... ............................................... . 118 3.1.2 Intentional dimension ............................................... 118 3.1.3 Structural dimension ................................................. 120 3.1.4 Communication dimension ....................................... 125 3.1.5 Dynamic dimension ............ .. ............... ................. .. .. 126

3.2 Applying the patterns .................................................... 128 4 Conclusion .......................................................................... 129

References.......................................................................... 130

Chapter S. 133 Design and behavior of a massive organization of agents Alain Cardon

1 Introduction ........................................................................ 133 2 The systems with particles ................................................. 136

2.1 The unsteady systems .................................................... 138 2.2 Operators of determination of the behavior

for an unsteady system .................................................. 140

xii Contents

3 The object approach: a very controlled process of construction and run of systems ..................................... 143

3.1 The object approach and the software engineering ....... 144 3.2 Objects and object-oriented design of systems ............. 145 3.3 Limits of the object approach ........................................ 147

4 Massive multi-agent systems ............................................. 148 4.1 Agents ............................................................................ 149 4.2 Nondeterminism and instability in massive

multi-agent systems ....................................................... 152 4.3 An agentification method for the massive

multi-agent systems ....................................................... 155 5 Analysis of the behavior of a massive agent

organization: the control problem ...................................... 162 5.1 The characterization of an agent organization ............... 163 5.2 The morphological space, the correspondent

of the space of phases for the MMAS ........................... 165 5.3 The organization of morphological agents assuring

the representation of the aspectual organization ........... 171 5.4 Characters of coherent groups ....................................... 175 5.5 Evocation agents and self-adaptability of the system .... 177

6 Entropy and equation of trajectory of MMAS ................... 180 6.1 Entropy .......................................................................... 181 6.2 Equation of trajectory: a reduction with regard

to the morphological analysis ........................................ 182 6.3 Validity of the state equations ....................................... 184 6.4 Degraded forms of the state equation ............................ 185

7 Conclusion .......................................................................... 185 References.......................................................................... 187

Chapter 6. 191 Developing agent-based applications with JADE F. Bergenti, A. Poggi, G. Rimassa, P. Turci and M. Tomaiuolo

1 Introduction ........................................................................ 191 2 JADE .................................................................................. 193

Contents xiii

2.1 Platform architecture ..................................................... 193 2.2 Agent architecture ......................................................... 196

3 LEAP ....................... ........... .. .. ... ............ .. ........................... 199 3.1 LEAP architecture ......................................................... 201

4 CoMMA ......... ... ...... ........ .... ..... .... .. .................................... 203 4.1 CoMMA architecture ...... .. .................... ...... ............... .... 204

5 Agentcities .......................................................................... 206 5.1 Network ......................................................................... 207 5.2 Service composition ................................................. ... .. 208

6 Conclusions ........................................................................ 210 Acknowledgments ............................................................ .. 211 References .......................................................................... 212

Chapter 7. 215 A collective can do better N.D. Monekosso and P. Remagnino

1 Introduction ........................................................................ 215 2 Insect behaviour can be inspiring ....................................... 217 3 Can nature be mimicked? ................................................... 219

3.1 Applying real ant behaviour to computational systems .......................................................................... 219

3.2 Solving classic optimisation problems .. .. ...................... 222 3.3 Telecommunications applications .............. .. ................. 223 3.4 Robot navigation applications .. .......... ........................... 224 3.5 Other robotic applications ............................................. 224 3.6 Image processing applications ....................................... 225

4 Combining reinforcement learning and synthetic pheromones ......................................................... 225

4.1 Reinforcement learning ....................... .......................... 225 4.2 Synthetic pheromones and Q-Iearning .......................... 226

5 Cooperative robotic transport.. ........................................... 232 6 Conclusions ........................ ....................... .. ....................... 232

References .......................... ... ......... .......... ........ ... ............... 233

xiv Contents

Chapter 8. 239 Coordinating multi-agent assistants with an application

by means of computational reflection A. Di Stefano, G. Pappalardo, C. Santoro, and E. Tramontana

1 Introduction .............................. ............. ... ....... ......... ... ... .... 240 2 The motivation for a multi-assistant architecture ............... 243

2.1 Example: extending a Web browser with assistant agents ...................................................... 244

3 The multi -agent reflective architecture ....................... .. ..... 246 3.1 Computational reflection ............................................... 247

3.1.1 Using lavassist.. ........................................................ 248 3.2 The architecture ............................ ................................. 251 3.3 Coordinator agent .......................................................... 253 3.4 Coordinator-assistants interactions .......... ... .................. . 257 3.5 A concrete example: an assistant that highlights

keywords for a Web browser. ........................................ 259 4 A case study: e-commerce assistants for a Web browser ... 264

4.1 User profiler assistant ...................... ............. ... ... ........... 265 4.2 Data extraction assistant.. .............................................. 268 4.3 Cart manager assistant.. .......... ....................................... 270

5 Concluding remarks .. ... ....... .... ........................................... 272 References .......................................................................... 274

Chapter 9. Learning by exchanging advice Eugenio Oliveira and Luis Nunes

279

1 Introduction ........................................................................ 279 2 Communicating to improve learning:

historical notes and review ................................................. 281 2.1 Early work on exchange of information

during learning ............................................................... 281 2.2 Recent related work ....................................................... 282

3 Advice exchange ................................................................ 283 3.1 Exchanging information during learning ....................... 283

Contents xv

3.1.1 What type of information? ...... ....................... .. .. .... ... 284 3.1.2 How to integrate this information with

the usual learning process? ....................................... 285 3.1.3 When should an agent request/accept information? .285 3.1.4 Where to get information? ........................................ 290

4 Experiments ........................................................................ 292 4.1 Predator-prey ................................................................. 294 4.2 Traffic control ........ ............ ............................................ 298 4.3 Learning algorithms ....................... ................. .. ............. 301

5 Results and discussion ........................................................ 304 5.1 Predator-prey ................................................................. 304 5.2 Traffic control ....................................... .. .... .. ......... ... ... .. 307

6 Conclusions and future work ............................................. 309 Acknowledgments .............................................................. 311 References .......................................................................... 311

Chapter 10. 315 Adaptation and mutation in multi-agent systems and beyond Ladislau BOioni and Dan Cristian Marinescu

1 Introduction ................................. .. .................................... . 315 2 A taxonomy ........................................................................ 317

2.1 Alternative names ............................ .. .. ... ... ... ........... .. .... 319 2.2 Classification criteria ..................................................... 319

2.2.1 The amplitude of the change: weak vs. strong mutability ........................................ 320

2.2.2 The granularity of mutation ...................................... 321 2.2.3 The continuity of interactions: runtime vs. stoptime 322 2.2.4 The initiator of the mutation ..................................... 322 2.2.5 Mutation technique ................................................... 323

2.3 A taxonomy of mutations ... .. ........ .................... ... .......... 324 2.4 Other classification approaches ... .... .... ..... ... ................ .. 325

3 A formal description of mutability ..................................... 325 3.1 Agent models and mutability .............. ...... .......... ........... 325 3.2 A multiplane state machine model of agent behavior ... 328

xvi Contents

3.3 Modelling agent behavior. ............................................. 329 3.3.1 Decomposition in the plane. expressing "change" ... 330 3.3.2 Expressing concurrency ............................................ 331

3.4 Mutation operators and invariance properties ............... 332 3.5 How useful are the invariance properties? .................... 334

4 A software engineering perspective on adaptive and mutable agents ........... .. ......................................... ....... 336

4.1 Adding new functionality to the agent .......................... 338 4.2 Removing functionality from an agent.. ........................ 341 4.3 Adapting to new requirements ...... .. ... .... ... ..................... 343 4.4 Splitting and merging agents ......................................... 346

5 Conclusions .................. ..................... ........... .. .... .. .... .. ........ 347 References .............................. ............................................ 349

Chapter 11. 355 Intelligent action acquisition for animated learning agents Adam Szarowicz, Marek Mittmann, laroslaw Francik

1 Introduction ........................................................................ 355 2 Current state of the art in automatic character animation .. 357

2.1 General animation architectures .................................... 357 2.2 Physics-based controllers ................. .. ........................... 361 2.3 Crowd simulation .......................................................... 362

3 Overview of other concepts ................................................ 363 3.1 Q-Iearning ...................................................................... 363 3.2 The agent's senses: collision detection and avoiding .... 364 3.3 Agent architectures ......... ............................................... 366

4 Implementation of the Q-Iearning ...................................... 367 5 System implementation and results .................................... 370

5.1 The framework .............................................................. 370 5.2 Results ............................................... ........... ................. 372 5.3 Alternative algorithms ............ ... .... ..... ... .......... .............. 378

6 Conclusions and summary ................................................. 378 Acknowledgements .. .. ........................ .... .... ........................ 380 References .......................................................................... 380

Contents xvii

Chapter 12. 387 Using stationary and mobile agents

for information retrieval and e-commerce Samuel Pierre

1 Basic concepts and background ......................................... 388 1.1 Agent and multi-agent systems .............. ... ........... ......... 388 1.2 Cooperation and communication mechanisms ........... .. . 390

1.2.1 Communication among agents .................................. 390 1.2.2 Cooperation among agents ........................................ 392

1.3 Mobile agent and mobile code ...................................... 394 2 Multi-agent architecture for information retrieval ............. 396

2.1 Mobile agent information retrieval.. .............................. 397 2.2 Characterization of the architecture ............................... 399 2.3 Experimental number application .................................. 408

2.3.1 Principles ofthe application ..................................... 408 2.3.2 Design choices and modifications

to the initial application ............................................ 409 2.4 Internet picture retrieval application ............................. 410

3 Implementation of the information retrieval architecture .. 411 3.1 Generic classes and interfaces ....................................... 411 3.2 Agents ............................................................................ 414 3.3 Implementation and testing environment ...................... 417

4 Evaluation of the information retrieval architecture .......... 419 4.1 Transportation measures ................................................ 419 4.2 Information retrieval scenarios ...................................... 422

5 Multi-agent architecture for product retrieval .................... 429 5.1 Description of the problem and general scenario .... .. .... 429 5.2 Solution and suggested algorithms ................................ 431 5.3 Architecture and agent structure .................................... 433 5.4 Implementation and performance evaluation ................ 436

6 Conclusion .......................................................................... 444 References .......................................................................... 446