a soft computing framework for intelligent agents: application to

A Soft Computing Framework for

Intelligent Agents: Application to

Software and Hardware Agents

Seung-Ik Lee

The Graduate School

Yonsei University

Department of Computer Science

and Industrial Systems Engineering

A Soft Computing Framework for

Intelligent Agents: Application to

Software and Hardware Agents

A Dissertation

Submitted to the Department of Computer Science

and Industrial Systems Engineering

and the Graduate School of Yonsei University

in partial fulfillment of the

requirements for the degree of

Doctor of Philosophy

Seung-Ik Lee

December 2001

This certifies that the dissertation of

Seung-Ik Lee is approved.

Thesis Supervisor: Sung-Bae Cho

Yoon-chul Choy

Yillbyung Lee

Sung-Bong Yang

Sang-Rok Oh

The Graduate School

Yonsei University

December 2001

c�

Copyright by Seung-Ik Lee, 2001.

All Rights Reserved.

iii

To my mother

and family

for their patience and understanding.

��?/ü<s�K�\�¦K�ÅÒ��

#Q Qm�ü<��7á¤\�>�s��7Hë�H�¦×¼wn�m��.

iv

CONTENTS

ABSTRACT xii

1 INTRODUCTION 1

1.1 Background and Motivation . . . . . . . . . . . . . . . . 2

1.1.1 Fundamental Concepts of Intelligent Agents . . . . 3

1.1.2 What is Soft Computing? . . . . . . . . . . . . . . 7

1.1.3 Need for New Soft Computing Framework . . . . 8

1.2 Research Objectives . . . . . . . . . . . . . . . . . . . . . 10

1.3 Organization of the Dissertation . . . . . . . . . . . . . . 12

2 RELATED WORKS 13

2.1 Conventional Soft Computing Frameworks . . . . . . . . 13

2.2 Analysis of Evolution . . . . . . . . . . . . . . . . . . . . 18

3 SOFT COMPUTING FRAMEWORK 21

3.1 Soft Computing Techniques and Frameworks . . . . . . . 22

3.1.1 Comparison of Soft Computing Techniques . . . . 22

3.1.2 Conventional Soft Computing Frameworks . . . . 25

3.2 Soft Computing Framework for Intelligent Agents . . . . . 27

v

3.3 Rule-based Systems . . . . . . . . . . . . . . . . . . . . . 30

3.3.1 Fuzzy Logic . . . . . . . . . . . . . . . . . . . . 31

3.3.2 Fuzzy Sets and Rule Base . . . . . . . . . . . . . 32

3.3.3 Fuzzy Reasoning and Defuzzification . . . . . . . 35

3.4 Evolutionary Algorithms . . . . . . . . . . . . . . . . . . 37

3.4.1 Genetic Algorithms . . . . . . . . . . . . . . . . . 38

3.4.2 Genetic Operations . . . . . . . . . . . . . . . . . 40

3.4.3 Learning with GA . . . . . . . . . . . . . . . . . 44

3.5 Analysis Methods . . . . . . . . . . . . . . . . . . . . . . 46

3.5.1 Evolutionary Activity Statistics . . . . . . . . . . 46

3.5.2 Observational Emergence . . . . . . . . . . . . . 50

3.5.3 Schema Analysis . . . . . . . . . . . . . . . . . . 52

4 APPLICATION TO A HARDWARE AGENT 54

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 56

4.2 Hardware Agent: KHEPERA . . . . . . . . . . . . . . . . 57

4.3 Rule-Based System: Fuzzy Logic Controller . . . . . . . . 59

4.4 Evolution of Behavior Controller . . . . . . . . . . . . . . 61

4.5 Evolution Analysis . . . . . . . . . . . . . . . . . . . . . 65

4.5.1 Adaptive Evolution . . . . . . . . . . . . . . . . . 65

4.5.2 Adaptive Behaviors . . . . . . . . . . . . . . . . . 71

4.5.3 Evolutionary Pathways to Solutions . . . . . . . . 76

4.6 Emergence Analysis . . . . . . . . . . . . . . . . . . . . 85

4.6.1 Turning Around . . . . . . . . . . . . . . . . . . . 85

4.6.2 Smooth Cornering . . . . . . . . . . . . . . . . . 89

4.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . 92

vi

5 APPLICATION TO A SOFTWARE AGENT 94

5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 96

5.2 Software Agent: Conversational Agent . . . . . . . . . . . 98

5.2.1 Dialogue Acts Classification . . . . . . . . . . . . 99

5.2.2 Knowledge Representation . . . . . . . . . . . . . 103

5.3 Behavior Controller : Rule-based Pattern Matcher . . . . . 106

5.4 Experimental Results . . . . . . . . . . . . . . . . . . . . 107

5.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . 109

6 CONCLUSION 111

BIBLIOGRAPHY 116

ABSTRACT (KOREAN) 128

vii

LIST OF FIGURES

1.1 Categories of intelligent agents [16]. . . . . . . . . . . . . 3

3.1 Conventional soft computing framework. . . . . . . . . . 25

3.2 Possible evolutionary pathways. . . . . . . . . . . . . . . 26

3.3 Proposed soft computing framework for agents. . . . . . . 28

3.4 Block diagram of a fuzzy controller. . . . . . . . . . . . . 31

3.5 Single point crossover. . . . . . . . . . . . . . . . . . . . 41

3.6 A single mutation. . . . . . . . . . . . . . . . . . . . . . . 42

3.7 General procedure of GA. . . . . . . . . . . . . . . . . . . 43

4.1 A framework for hardware agents. . . . . . . . . . . . . . 55

4.2 (a). Khepera mobile robot (b). IR sensors and motors [46]. 57

4.3 Position of some parts of the robot [46]. À LEDs. Á Se-

rial line (S) connector. Â Reset button. Ã Jumpers for the

running mode selection. Ä Infra-Red proximity sensors. Å

Battery recharge connector. Æ ON-OFF battery switch. Ç

Second reset button. . . . . . . . . . . . . . . . . . . . . . 58

4.4 Gene representation of an individual. . . . . . . . . . . . . 61

4.5 Three out of five membership functions need to be encoded. 62

viii

4.6 Encoding of a rule. . . . . . . . . . . . . . . . . . . . . . 62

4.7 Environment. . . . . . . . . . . . . . . . . . . . . . . . . 64

4.8 (a). Fitness changes. (b). Trajectory of the best individual. 65

4.9 Fuzzy sets obtained by evolution. . . . . . . . . . . . . . . 66

4.10 Activity distribution function �� . . . . . . . . . . . . 68

4.11 (a). Diversity � �� . (b). Total cumulative evolutionary ac-

tivity �� . (c). Mean cumulative evolutionary activity�� . (d). New activity �� . . . . . . . . . . . . . 69

4.12 Log-log plot of the component activity distributions for the

fuzzy model and its neutral shadow. . . . . . . . . . . . . 70

4.13 Evolutionary activities of salient waves. . . . . . . . . . . 72

4.14 Behaviors of S � and corresponding data. . . . . . . . . . . 73

4.15 Behaviors of S �� S � and corresponding data. . . . . . . . . 75

4.16 Behaviors of S �� S �� S and corresponding data. . . . . . 77

4.17 Analysis of schemata S ! �#" and B ! �#" . . . . . . . . . . . . . 80

4.18 Analysis of the schemata S ! �$" and B ! �$" . . . . . . . . . . . 81

4.19 Analysis of schemata S !#% " and B !#% " . . . . . . . . . . . . . 82

4.20 The lineage of the best individual. . . . . . . . . . . . . . 84

4.21 &('*) of the three first order structures: (a). &+'*),�#- ��. . (b).&+'*),�#- �%. . (c). &+'*),�#- �/ . . . . . . . . . . . . . . . . . . . . 86

4.22 Analysis of &+'*) ��. of turning around. . . . . . . . . . . . . 87

4.23 &('*) � of turning around. . . . . . . . . . . . . . . . . . . . 87

4.24 Analysis of &+'*) � of turning around. . . . . . . . . . . . . 88

4.25 &('*) of the two first order structures of smooth cornering:

(a). &+'*),�#- ��. (b). &+'*),�#- �/ . . . . . . . . . . . . . . . . . . 89

4.26 Analysis of &+'*) ��. of smooth cornering. . . . . . . . . . . 90

ix

4.27 Smooth cornering. . . . . . . . . . . . . . . . . . . . . . . 90

4.28 Analysis of &+'*) � of smooth cornering. . . . . . . . . . . . 91

5.1 A framework for software agents. . . . . . . . . . . . . . 95

5.2 Structure of a conversational agent. . . . . . . . . . . . . . 98

5.3 Automata for location question. . . . . . . . . . . . . . . 101

5.4 Dialogue acts classification using subsumption architecture. 102

5.5 Internal representation of script. . . . . . . . . . . . . . . 106

5.6 Overall response matching procedure. . . . . . . . . . . . 108

5.7 Location question 1. . . . . . . . . . . . . . . . . . . . . . 108

5.8 Location question 2. . . . . . . . . . . . . . . . . . . . . . 108

x

LIST OF TABLES

4.1 Rules of the best individual. . . . . . . . . . . . . . . . . 66

4.2 Salient rules. . . . . . . . . . . . . . . . . . . . . . . . . 74

4.3 Test rule set for S � . . . . . . . . . . . . . . . . . . . . . . 74

4.4 Test rule set for S . . . . . . . . . . . . . . . . . . . . . . 76

4.5 Schemata of the salient rules. . . . . . . . . . . . . . . . . 78

4.6 Schemata of the best individual’s rules. . . . . . . . . . . . 78

4.7 - ��. , - �% . , and - �/ . . . . . . . . . . . . . . . . . . . . . . . . 85

5.1 Dialogue acts. . . . . . . . . . . . . . . . . . . . . . . . . 100

5.2 A part of BNF grammar for the query-response script database.104

5.3 Comparison functions. . . . . . . . . . . . . . . . . . . . 105

5.4 A part of script. . . . . . . . . . . . . . . . . . . . . . . . 105

5.5 Score assignment. . . . . . . . . . . . . . . . . . . . . . . 107

xi

ABSTRACT

A Soft Computing Framework forIntelligent Agents: Application to Software and

Hardware Agents

Seung-Ik Lee

Dept. of Computer Science and

Industrial Systems Engineering

The Graduate School

Yonsei University

The old ways of interactions between human and computer require

more time and effort of users and make untrained users to meet more diffi-

culties. Recent research on agents has opened a new way for solving those

problems, especially utilizing soft computing techniques.

Two main frameworks are used to apply soft computing techniques to

the construction of agents. One is to apply each soft computing technique

independently. It has some drawbacks in determining the internal param-

eters of the technique used. The other one is to combine two or more soft

computing techniques to relieve the difficulties of tuning internal param-

eters. Especially, the combination of an evolutionary algorithm and other

techniques has been extensively attempted.

Despite the success of the conventional framework, it has at least three

shortcomings. First, it cannot guarantee that evolved solutions are from

xii

adaptation, since evolutionary phenomena contain not only adaptation but

also others such as chances, necessity, and random genetic drift. Second,

it cannot show what, when, how, and why some genes have successfully

evolved into good solutions while the others not. These questions are crit-

ical to show causes of adaptation and evolution. Finally, it is not able to

analyze the evolved solutions. Evolution necessarily causes the changes of

simple structures into complex ones that may be viewed as emergent behav-

iors or emergent properties. However, the conventional frameworks cannot

tell whether certain behaviors are emergent because they do not have any

formal definition of emergence.

This dissertation proposes a soft computing framework for intelligent

agents to overcome the shortcomings of the conventional frameworks. The

framework consists of two parts: construction and analysis part. In the

construction part, the combination of rule-based systems and evolutionary

algorithms is used to construct an intelligent agent. In the analysis part,

evolutionary activity statistics, schema analysis, and observational emer-

gence are applied to the analysis of the evolved agent in both evolutionary

and behavioral perspectives.

Evolutionary activity statistics is a quantitative measure for adaptive

evolution and can tell objectively and quantitatively whether an evolution is

adaptive. Schema analysis is to identify evolutionary pathways to solutions.

Observational emergence identifies emergent behaviors of the evolved con-

troller.

To show the usefulness of the framework, this dissertation applies it to

two agents: software and hardware agents, which are the two main fields of

agent research.

xiii

In the first case study, the framework is applied to the construction of a

behavior controller for a hardware agent. The hardware agent called Khep-

era is a kind of behavior-based mobile robot. Eight proximity sensors are

used as input to the behavior controller and two control values are produced

from the controller to drive the two motors of the robot. Fuzzy logic, a kind

of rule-based systems, is used for the behavior controller, and a genetic

algorithm tunes the internal parameters of the fuzzy behavior controller.

Analysis is performed on the evolved controller and the evolutionary pro-

cess using the framework.

In the second case study, the framework is applied to the construction

of a behavior controller for a software agent. The software agent is a con-

versation agent that can act as a virtual representative of a web site interact-

ing with visitors using natural language. The agent consists of three main

components: dialogue act categorization, structured pattern matching, and

knowledge construction and representation. Dialogue acts (DAs) are clas-

sified by automata that accept a sequence of keywords to identify user’s

intentions. In addition, subsumption architecture controls the interactions

among DA analysis modules to make the classification of DA more effec-

tive. We utilize pattern matching techniques rather than conventional nat-

ural language processing techniques to match users’ queries with a knowl-

edge base.

The two case studies show that the framework can be effectively ap-

plied to the construction of controllers for intelligent agents. In addition,

the analysis on the evolutionary process presents that the solutions have re-

sulted from the adaptability of each component that constitutes the solution

rather than from chances or other evolutionary phenomena. The analysis

xiv

on the behaviors of the evolved controllers shows that the controller has

obtained emergent behaviors through the evolutionary process. Especially,

the emergent behaviors have resulted from the interactions of lower level

components of behaviors.

Consequently, we have confirmed that the framework can be effectively

applied to the construction of controllers for intelligent agents. Particularly,

it has advantages in analyzing the behaviors of an evolved controller and in

analyzing the evolution.

Key Words: Adaptability, Conversational Agent, Evolutionary Activity Statis-

tics, Evolutionary Algorithm, Fuzzy Logic, Intelligent Agent, Mobile Robot,

Observational Emergence, Schema, Soft Computing Framework.

xv

CHAPTER 1

INTRODUCTION

The “information highway” is presenting us with computer-based works

of increased complexity, requiring new ways of interactions between hu-

man and computer. In existing ways, computer-based systems are guided

by the direct manipulation of users. However, the increase of the amount of

information requires more time and effort of users and present users with

more difficulties. Recent research on agents has opened a new way for

solving those more complex works.

Though there is no commonly accepted answer for what an agent is,

it can be generally regarded as a computer-based system that can reason

and act in order to achieve users’ goals while interacting with not only

users but also environments. For the fulfillment of users’ goals, an agent

needs accurate recognition of current situation and reasoning. Because of

the complexity of interactions with users or environments, they are typi-

cally ill-defined, difficult to model, and with large-scale solution spaces. In

these cases, precise models are impractical, too expensive, or non-existent,

1

1. INTRODUCTION

because the relevant available information is usually the form of empirical

prior knowledge and input-output data representing instances of the sys-

tem’s behavior. Fortunately, soft computing technologies provide us with a

set of flexible computing tools to perform these approximate reasoning and

search tasks with such imperfect information.

This dissertation presents a soft computing framework for intelligent

agents by augmenting conventional soft computing frameworks and applies

it to two case studies, software agent and hardware agent, to show its use-

fulness.

The organization of this chapter is as follows: In Section 1.1, we intro-

duce fundamental concepts of intelligent agents, describe soft computing

techniques briefly, and motivate the need for a new framework by describ-

ing the problems and limitations of the previous soft computing frame-

works. In Section 1.2, we discuss the research objectives of this study.

Finally, Section 1.3 outlines the organization of the rest of this dissertation.

1.1 Background and Motivation

In this section, we first describe on the definition of agent and what

properties an agent should have. Second, we describe what soft comput-

ing is and why it is necessary for intelligent agents. Finally, we mention

the merits and demerits of conventional soft computing frameworks, and

discuss the need for a new soft computing framework to overcome the de-

merits of the conventional frameworks.

2

1. INTRODUCTION

1.1.1 Fundamental Concepts of Intelligent Agents

Before proceeding further on the concepts of intelligent agents, we first

present a category of agents, which almost agent researchers can accept.

Zarnekow categorized agents at a highest level as in Fig. 1.1 [16].

Intelligent Agents

Human Agents

(e.g. travel agent)

Hardware Agents

(e.g. robot)

Software Agents

Information Agents

Cooperation Agents

Transaction Agents

Fig. 1.1: Categories of intelligent agents [16].

All the categories have the common feature that they independently per-

form tasks, which require specialized knowledge or consists of many time

intensive steps, on behalf of their contracting party or user.

Definitions of Intelligent Agents

Though it is commonly accepted that hardware and software agents

should perform certain tasks for their users that they cannot undertake them-

selves because of insufficient time or lack of knowledge, it has not yet been

possible to agree on a generally accepted, comprehensive, definition of an

intelligent agent.

However, many researchers have tried to define what an agent is [16,

29, 72]. Brenner et al. [16] referred to an agent as “an agent always re-

3

1. INTRODUCTION

quires a certain amount of intelligence to perform its tasks. Consequently,

one refers to intelligent agents. A non-intelligent agent can be considered

as any traditional software program, because even traditional programs per-

form a specific task and provide their users with a direct time saving. Only

the intelligence permits an agent to perform its task largely autonomously

and to require intervention from the user only for important decisions.” An

agent that cannot process tasks independently is not much useful because

its user should intervene in the process and, thus, gain only a minimal time

saving. Consequently, autonomous processing is one of both important cri-

teria for intelligent agents and main differences between intelligent agents

and traditional software programs.

An agent must interact with environments to achieve its goals and be

capable of gathering information on its environment. Making decisions

based on this information, it then initiates specific actions. Taking these

considerations into account, Brenner et al. [16] concluded that “intelligent

software agents are defined as being a software program that can perform

specific tasks for a user and possesses a degree of intelligence that permits it

to perform parts of its tasks autonomously and interact with its environment

in a useful manner.”

Shoharn [72] defined a software agent as “a software entity which func-

tions continuously and autonomously in a particular environment, often

inhabited by other agents and processes.” The continuity and autonomy

ensures that an agent is capable of carrying out activities in a flexible and

intelligent manner that is responsive to changes in the environment without

human intervention.

Franklin and Graesser [29] defined an autonomous agent as “a system

4

1. INTRODUCTION

situated within and part of an environment that senses the environment and

acts on it, over time, in pursuit of its own agenda and so as to effect what it

senses in the future”

Although the above definitions are mainly concerned with software

agents, many concepts apply equally to hardware agents. As there is no

commonly accepted definition of an intelligent agent, we refer to an in-

telligent agent as having autonomy and interaction with its environment,

mainly based on Brenner’s definition. In addition, we use the two terms,

intelligent agent and agent, interchangeably unless we refer to one of them

specifically.

Properties of Intelligent Agents

This section describes in detail some of important characteristics of in-

telligent agents to obtain an understanding of the tasks and functions of

them. One thing to note here is that not every agent must have all the listed

characteristics. Actually, there are many systems with different complex-

ities: A simple agent with very low complexity may have one or two of

the characteristics whereas agents with high complexity can have all the

characteristics in the ideal situation.0 Autonomy: One of important differences between agents and other

non-agent entities is the capability of an agent to follow its goals

autonomously. It refers to the principle that agents can operate on

their own without the need for human guidance, even though this

would sometimes be invaluable. Hence, agents have individual inter-

nal states and goals, and they act in such a manner as to meet its goals

5

1. INTRODUCTION

on behalf of its user [62].0 Reactivity: Reactivity designates that an agent must be capable of

reacting appropriately to influences or information from its environ-

ment. This environment can consist of other agents, human users, ex-

ternal information sources, or physical objects. Reactivity forms one

of the fundamental requirements for an intelligent agent and, there-

fore, all agents should support it to some extent. The agent must have

suitable sensors or possess its own internal model of its environment

in order to be able to react to changes in the environment [16].0 Reasoning and learning: Reasoning capability puts an agent in the

position of being able to observe its environment and to make specific

decisions when changes occur in this environment. AI techniques,

such as rule-based systems, knowledge-based systems, or neural net-

works are well suited for reasoning. These include an artificial evo-

lution model in which agents create new generations of agents that

have increasingly complex reasoning mechanisms [9].0 Proactivity: Proactivity designates that agents do not simply act in

response to their environment, they are able to exhibit goal-directed

behavior by taking the initiative [88]. It is a level above the reactivity.

The capability for an agent itself to take the initiative requires that

the agent have well-defined goals or even a complex goal system.

However, true proactive behavior occurs only to a very limited extent

with complex goal systems.0 Social ability: An agent often interacts with its environment, espe-

6

1. INTRODUCTION

cially humans or other agents via some kind of communication lan-

guages. This characteristic designates two kinds of properties. One is

the communications ability with humans or other agents. The other is

the cooperation ability with other agents. However, the latter does not

always require certain kind of communication languages [16, 62, 88].

If the listed characteristics are compared with any existing agent, it is

apparent that few of them, currently available or in development, have all

or most of the properties. In addition, traditional algorithmic approaches

cannot easily solve the listed properties, because the properties are difficult

to model, ill defined, and require large-scale solution space. Here comes

the need for soft computing techniques in order for software and hardware

agents to overcome the complexity.

1.1.2 What is Soft Computing?

Soft computing is a recently coined term describing the symbiotic use of

many emerging computing disciplines [18, 32, 45, 47, 79]. Zadeh defined it

as “... in contrast to traditional, hard computing, soft computing is tolerant

of imprecision, uncertainty, and partial truth.” [93]

Soft computing encompasses computational techniques such as neural

nets, fuzzy systems, probabilistic reasoning (e.g., Bayesian networks), and

evolutionary algorithms (e.g., genetic algorithms). The common denomi-

nator of these technologies is their departure from classical reasoning and

modeling approaches that are usually based on Boolean logic, analytical

models, crisp classifications, and deterministic search. These classical ap-

proaches describe the system to be modeled by complete and precise infor-

7

1. INTRODUCTION

mation. The state of the system can be stated to be binary truth-values by

formal reasoning process.

Real world problems, however, are typically ill defined, difficult to

model, and with large-scale solution spaces, and therefore precise models

are impractical, too expensive, or non-existent. The relevant available in-

formation is usually limited to empirical knowledge and input-output data

representing the system’s behaviors. Consequently, approximate reasoning

systems capable of handling such imperfect information are required.

Soft computing techniques can handle imperfect information of com-

putationally hard problems that do not seem to be solved by classical algo-

rithmic approaches. Soft computing is expected to show good performance

in the processing of linguistic information, processing of pictorial informa-

tion such as classifying characters, filtering information from textual and

pictorial databases, analogical reasoning, and so on.

1.1.3 Need for New Soft Computing Framework

There are two main frameworks in applying soft computing techniques

to the construction of agents. One is to apply each soft computing technique

independently and the other is to combine two or more techniques.

The former framework has advantages in its methodological simplicity

and facilities in construction if applied to a relatively simple and tiny prob-

lem. However, it has some drawbacks in determining the internal parame-

ters of the techniques because it requires expert-level knowledge and needs

more time and effort depending on the problems and the technique used.

For example, it is well known fact that to determine the optimal topology

8

1. INTRODUCTION

and weights of an artificial neural network is hard, time-consuming work.

The latter framework is proposed to solve the problems of the former

framework. This framework combines two or more different soft comput-

ing techniques to relieve the difficulties of determining internal parameters.

Especially, the combination of an evolutionary algorithm and other soft

computing technique has been focused very much, where the evolution-

ary algorithm tunes the internal parameters of its partner through evolution.

This framework has advantages in that it requires relatively less expert-level

knowledge and expertise and can show better performance because of the

global searching capabilities of the evolutionary algorithm. Much of previ-

ous work has shown good results in finding good behavior controllers for

agents by adopting this framework [11, 19, 31, 43, 54, 60, 69, 77].

Despite the success of the combination of soft computing techniques

and evolutionary algorithms, it has at least three shortcomings. First, it

cannot guarantee that evolved solutions are from adaptive evolution. In

general, evolutionary phenomena contain not only adaptive evolution but

also others such as chances, necessity, and random genetic drift [20, 58, 84].

Therefore, it cannot be said that the solutions have resulted from adaptive

evolutionary process. In other words, the same good solutions as previous

ones may not be obtained anymore in the follow-up experiments even if

experimental conditions are the same. Therefore, to prove or show that the

evolved solutions have resulted from adaptive evolution has great scientific

importance on the justification of the experiments and evolution.

Second, it cannot show evolutionary pathways to the solutions [19, 27,

37, 55]. Here, we mean by evolutionary pathways that what, when, how,

and why some genes have successfully evolved into the solutions while the

9

1. INTRODUCTION

others not. These questions are critical to show the causes of adaptation

and evolution.

Finally, it is not able to analyze the evolved solutions. Evolution neces-

sarily causes the changes of simple structures into complex ones that may

be viewed as emergent behaviors or emergent properties [41, 73]. However,

the second framework cannot tell whether certain behaviors are emergent

because it does not have any formal definition of emergence.

1.2 Research Objectives

In the above discussion, we have identified some problems of the two

conventional frameworks and motivated the need for a new soft computing

framework to construct a behavior controller for an agent. This section

summarizes the research objectives of this dissertation with the details of

how these objectives are accomplished deferred to a later chapter:0 New soft computing framework: The two conventional frameworks

described above have some drawbacks, especially in the perspective

of evolution. In this dissertation, a soft computing framework is pro-

posed to show the roles and effects of evolutionary algorithms during

the evolution of intelligent agents.0 Quantification of adaptive evolution: In general, it is hard to say that

evolution has been adaptive during the evolutionary process. How-

ever, without any confirmation that the evolution has been adaptive,

there is no guarantee that follow-up experiments will yield the same

or similar results as previous ones, even though conditions are the

10

1. INTRODUCTION

same. The quantification of adaptability of evolution helps us show

more scientifically and objectively that the evolution has been adap-

tive during the evolutionary process.0 Adaptive behaviors: If a certain level of adaptability is maintained,

evolution can produce components that have adaptive behaviors. These

adaptive components are expected to contribute to the formation of

good solutions. Quantification of adaptability will help to show the

reasons that some components are more adaptive than others are.0 Evolutionary pathways to solutions: Illustrating evolutionary path-

ways to solutions is to show how the solutions are evolved. In other

words, it is used to show that the solutions are not from chances or

other genetic phenomena but from the adaptations to the environ-

ment.0 Illustration of emergence: Evolution has the capability of creating

new complex structures from simple structures. This kind of con-

cept, i.e., the creation of new structures or properties, is called emer-

gence. In natural evolutionary systems, emergence has been continu-

ously created as lives evolve, and no one denies this fact. In artificial

evolutionary systems, much work has insisted that behaviors of the

evolved systems are emergent without any formal definition of emer-

gence. Consequently, the insistence cannot be commonly accepted.

In this dissertation, we analyze the behaviors of evolved agents and

show that their behaviors are emergent relying on a formal definition

of emergence.

11

1. INTRODUCTION

1.3 Organization of the Dissertation

Chapter 2 discusses the results and problems of previous works in the

perspective of soft computing frameworks. In Chapter 3, a soft computing

framework for agents is proposed and each component of the framework is

described in detail.

In Chapter 4, a hardware agent system is developed to show the use-

fulness of the proposed framework. In this chapter, the framework is ap-

plied to the construction of a behavior controller for a mobile robot. After

evolving the behavior controller, we analyze the evolution and the evolved

controller’s behaviors with the framework.

In Chapter 5, a software agent system is developed to show the useful-

ness of the framework in the paradigm of software agents. The software

agent, called conversation agent, is to answer to questions from users with

natural language interface. It can process both Korean and English. Its

role is to introduce a web site to users as a virtual representative. The be-

havior controller of the conversational agent is evolved by an evolutionary

algorithm and analyzed with the framework.

We conclude in Chapter 6 by summarizing the contributions of this dis-

sertation and discussing promising directions for further works.

12

CHAPTER 2

RELATED WORKS

In this chapter, we discuss related works in topics close to this disser-

tation. Section 2.1 presents discussions about conventional approaches to

the combination of evolutionary algorithms and other soft computing tech-

niques. Here, we discuss mainly about the combination of fuzzy logic and

evolutionary algorithms, and the combination of neural networks (NNs) and

evolutionary algorithms, because they are the most commonly used types

of combination of soft computing techniques. Section 2.2 discusses related

work on emergence and measuring adaptive evolution.

2.1 Conventional Soft Computing Frameworks

Over the past ten years, an increasing number of hybrid algorithms have

been proposed, in which two or more soft computing technologies have

been integrated to improve the overall algorithm performance.

Many researchers have explored the use of genetic algorithms (GAs)

13

2. RELATED WORKS

to tune fuzzy logic controllers (FLC). These methods differ mostly in the

order or the selection of the various FLC components that are tuned, i.e.,

term sets, rules, and scaling factors [11, 31, 37, 38, 49, 53, 54, 55, 60, 69].

Bonarini [11] presented some approaches based on evolutionary rein-

forcement learning algorithms, which were able to evolve in real-time fuzzy

models that control behaviors. He discussed an application where an agent

learns how to adapt its behavior to the different behaviors of the other agents

it is interacting with, and another application where a group of agents co-

evolve cooperative behaviors by using explicit communication to propose

the cooperation and to distribute reinforcement to the others.

Fukuda and Kobota [31] proposed a robotic system with structured in-

telligence. They focused on a mobile robotic system with a FLC and pro-

posed a sensory network to allow the robot to perceive its environment.

An evolutionary algorithm was used to improve the robot’s performance.

They showed the effectiveness of the proposed method through computer

simulations of collision avoidance and path-planning problems.

Herrera et al. [38] directly tuned each rule used by the FLC. They used

a real encoding for a four-parameter characterization of a trapezoidal mem-

bership value used in the rule antecedent and consequent. The population

was the concatenation of all rules. A customized crossover operator was

also proposed. The fitness function was a sum of quadratic errors.

Kinzel et al. [49] tuned both rules and term sets. They departed from

the string representation and used a matrix to encode the rule set. They also

proposed customized crossover operators that were similar to the two-point

crossover for string encoding. They first initialized the rule base according

to intuitive heuristics, used GAs to generate better rule base, and finally

14

2. RELATED WORKS

tuned the membership functions of the best rule base.

Lee and Takagi [53] tuned the rule base and the term sets. They used

a binary encoding for each three-tuple characterizing a triangular mem-

bership distribution. Each chromosome represented a Takagi-Sugeno rule,

concatenating the membership distributions in the rule antecedent with the

polynomial coefficients of the consequent.

Liu et al [55] proposed a novel FLC, called linguistic hedge FLC, to

simplify the membership function constructions and the rule developments.

The design methodology of linguistic hedge FLC was a hybrid model based

on the concepts of the linguistic hedges and the GAS. The linguistic hedge

operators were used to adjust the shape of the system membership func-

tions dynamically, and ran speed up the control result to fit the system de-

mand. Genetic algorithms were adopted to search the optimal linguistic

hedge combination in the linguistic hedge module.

Melin and Castillo [60] applied different hybrid approaches for control-

ling the battery charging process. The hybrid approaches combined soft

computing techniques to achieve the goal of controlling the temperature

of the battery during the electrochemical charging process. They reduced

the time required for charging a battery with the use of fuzzy logic, NNs,

and GAs. In the neuro-fuzzy-genetic approach, NNs were used for mod-

eling the electrochemical process, fuzzy logic was used for controlling the

process, and GAs were used to optimize the fuzzy system.

Seng [69] proposed a neuro-fuzzy logic controller where all of its pa-

rameters could be tuned simultaneously by GA. The structure of the con-

troller was based on the radial basis function NN with Gaussian member-

ship functions. They used a flexible position coding strategy of the neuro-

15

2. RELATED WORKS

fuzzy logic controller parameters to obtain near optimal solutions.

As explained above, many researchers have shown that GAs can tune

the FLC parameters to produce appropriate FLC rules, shapes, and centers

of fuzzy membership functions. However, they did not show whether their

evolutionary processes were adaptive. They also did not show evolutionary

pathways to the obtained solutions.

There are many forms in which GAs can be used to synthesize or tune

neural network (NN): to evolve the network structure, to evolve a set of

weights for a given structure, and to evolve the reward function. Encoding

both NN structure and parameters requires too large GA chromosome such

that the GA cannot search the global space efficiently. Therefore, the above

approaches are usually mutually exclusive in most cases.

Chen and Miikkulainen [19] applied a NN to music composition to

generate melodies. An evolutionary algorithm was used to find a NN that

maximizes the chance of generating good melodies. Composition rules on

tonality and rhythm were used as a fitness function for the evolution. They

observed that the model learns to generate melodies according to these rules

with interesting variations.

Eccles et al. [25] developed a virtual ecosystem using GAs, artificial

NNs, and fuzzy systems. The ecosystem simulated and regulated the mo-

tion and interactions of computer animated agents in a virtual environment.

Within the ecosystem, each agent had its own NNs that govern its mo-

tion strategy so that the creature could learn to perform different behaviors,

such as searching for food and evading predators. The inputs of the NNs

were connected with creatures’ eye-like sensors and the outputs of the net-

works were attached to creatures’ legs to drive the creatures. The NNs

16

2. RELATED WORKS

were evolved using GAs, including single tournament selection, two-point

crossover, and one point mutation. Augmented fuzzy cognitive maps regu-

lated the overall operation of the environment and allowed each creature to

switch between different NNs for finding food and avoiding predators.

Floreano and Mondada [27] evolved a real, neural-network driven mo-

bile robot. The whole evolutionary process took places entirely on a real

robot without human intervention. Although the experiments tackled a sim-

ple task of navigation and obstacle avoidance, They showed a number of

emergent phenomena that are characteristic of autonomous agents.

Hunter and Chiu [43] discussed the design of NN and FLCs using GAs,

for real-time control of flows in sewerage networks. The soft controllers

operated in a critical control range, with a simple set-point strategy gov-

erning ‘easy’ cases. A GA designed controllers and set-points by repeated

application of a simulator. They also provided a comparison between NN,

fuzzy logic and benchmark controller performance. They concluded that

NN and FLCs had comparable performance, though NNs could be success-

fully optimized more consistently.

Tan et al. [77] applied genetic programming to the construction of NNs.

They described a genetic programming approach that constructs neural

logic networks from training examples. Neural logic network, unlike or-

dinary networks, allows representation of a great multitude of logic opera-

tions used in human decision-making. Their method enabled the construc-

tion of the network from scratch, but also preserved the semantic contents

of the network after training.

The previous researchers discussed so far showed that soft comput-

ing techniques can be tuned to produce appropriate solutions for the tasks.

17

2. RELATED WORKS

However, they did not show that their evolved solutions had resulted from

the adaptabilities to the problems. Furthermore, it is still necessary to reveal

the control mechanisms of the behavior controller obtained by evolution to

understand how the controller works.

2.2 Analysis of Evolution

Recently, the concept of emergence [21, 23, 61] has been focused as a

result of a lot of research in non-linear dynamics, artificial life, complex

systems, and behavior-based robotics. Emergence of a system, in a broad

way, is the properties or behaviors that cannot easily be predicted from its

internal properties. Examples of these phenomena are the flocking behavior

in simulated birds from a set of three simple steering behaviors [67], some

patterns in the game of life [10], and the “highway” pattern by the artificial

Langton ant [75]. Despite the broadness of emergence phenomena, there is

no unified agreement on what emergence is.

One of the first definition on emergence was made in 1923 by Morgan

who defined: “emergence is the denomination of something new which

could not be predicted from the elements constituting the preceding con-

dition,” “the product is not a mere sum of the separate elements,” or “the

creation of new properties” [26, 61]. Bedau proposed weak emergence in

contrast to strong emergence [5]. According to his definition, properties

or behaviors of a system are weakly emergent if and only if they can be

derived from the dynamics that governs the time evolution of the system’s

microstates, only by simulation. Ronald proposed a three-step emergence

test [68]. The three steps are design, observation, and surprise. He drew an

18

2. RELATED WORKS

analogy with the concept of intelligence and the Turing test. Baas [3] de-

fined two emergence: deducible emergence if there is deductional or com-

putational process or theory, and observational emergence if it cannot be

deduced.

A lot of literature in evolutionary biology, artificial life, and evolu-

tionary computation have revealed that good solutions for a given prob-

lem are not automatically obtained just because evolutionary algorithms

are used, as there are many evolutionary phenomena such as random ge-

netic drift, necessity, and adaptation that can affect the formation of the

solutions [20, 84]. The good solutions obtained might have not resulted

from adaptive evolution but from other unexpected evolutionary phenom-

ena. In this case, the same results would not be achieved by follow-up

evolutionary experiments if some set-ups such as mapping from genotype

to phenotype [34, 78] are not proper.

The capacity to produce good solutions via evolution, evolvability, can

be defined in various ways as “genome’s ability to produce adaptive vari-

ants when acted on by the genetic system” by Wagner and Altenberg [82],

“capacity to generate heritable phenotypic variation” by Kirshner and Ger-

hart [50], or “capacity to create new adaptations, and especially new kinds

of adaptations, through the evolutionary process” by Bedau [6]. To obtain

good solutions robustly, the evolvability should be maintained during the

evolutionary process. Under this condition, good candidate solutions can

contribute to the formation of good solutions.

A promising way to measure evolvability was proposed by Bedau and

Packard [6]. This method measures evolvability by quantifying adaptive

phenomena in artificial and natural systems using evolutionary activity statis-

19

2. RELATED WORKS

tics. The evolutionary activity statistics and its related diagrams are use-

ful to show evolutionary dynamics and, therefore, to assess the level of

evolvability in a system, which is a key step toward making evolution-

ary systems that are highly evolvable. The statistics has been applied to

a variety of evolving systems for different purposes. They applied it to vi-

sualizing adaptive evolutionary phenomena [6, 7, 8], studying punctuated

equilibrium dynamics in evolution [71], identifying long-term evolutionary

trends [56], and classifying evolutionary dynamics [7].

20

CHAPTER 3

SOFT COMPUTING

FRAMEWORK

This chapter presents a soft computing framework for intelligent agents

and describes each component of the framework in detail. We start by

describing conventional soft computing techniques and frameworks in sec-

tion 3.1. Section 3.2 presents the proposed framework and explains it fo-

cusing on the augmented analysis part.

In subsequent sections, we describe each soft computing technique that

is a component of the proposed framework. In section 3.3, we describe rule-

based systems briefly and compare the merits and demerits of the system

with other soft computing techniques. Next, we describe fuzzy logic as a

kind of rule-based system.

In section 3.4, we overview evolutionary algorithms and describe ge-

netic algorithms in detail, which is one of the components of our frame-

work. Section 3.5 describes three methods used to analyze the evolution

21

3. SOFT COMPUTING FRAMEWORK

and emergent behaviors of intelligent agents: evolutionary activity statis-

tics, observational emergence, and schema analysis.

3.1 Soft Computing Techniques and Frameworks

Soft computing encompasses several computational techniques such as

neural networks, fuzzy systems, probabilistic reasoning, and evolutionary

algorithms. All the techniques have in common departure from classi-

cal reasoning and modeling approaches that are usually based on Boolean

logic, crisp classifications, and deterministic search.

In this section, we describe soft computing techniques briefly and dis-

cuss their merits and demerits. We also describe conventional soft comput-

ing frameworks and their shortcomings.

3.1.1 Comparison of Soft Computing Techniques

Soft computing techniques are particularly effective in search and opti-

mization when the underlying search space is large, multimodal, and when

the characteristics of the search space are not well understood. It is under

these conditions that classical reasoning and modeling approaches become

impractical [12].

Fuzzy logic approaches are based on a fuzzy-valued representation of

uncertainty and imprecision. Typically, they use linguistic variables to rep-

resent different information granularities, and triangular-norms to propa-

gate the fuzzy boundaries of such granules. The basic mechanism used in

fuzzy reasoning systems for inference is the generalized modus-ponens [92].

22


Fuzzy logic translates and embeds empirical, qualitative knowledge about

the problem to be solved into reasoning systems that can perform approx-

imate pattern matching and interpolation. Though fuzzy logic has advan-

tages in that expert knowledge can be easily represented and, therefore,

understandable, it does not have adaptation or learning features, since it

lacks the mechanism to extract knowledge from existing data. Therefore,

fuzzy logic is difficult to apply when expert knowledge of a give problem

is insufficient.

On the other hand, neural network has learning capability as one of its

typical characteristics. With enough training data, the weights inside the

neural network are adjusted by comparing the output of the network to the

desired answer. Because of this training capability, neural networks have

been applied to many problems such as pattern classification, pattern cat-

egorization, and control of vehicles. However, neural networks also have

drawbacks. First, training neural networks is not straightforward and can be

tricky. It requires much time and effort because no optimal solution exists

in determining the internal parameters of the neural network. Second, the

trained neural network is usually a kind of black-box. In other words, it is

hard to understand just why the trained neural network works. Finally, neu-

ral network must have training data. In addition, test data are also required

if the performance of the trained neural networks are to be measured.

Another soft computing technique is probabilistic reasoning. Proba-

bilistic reasoning such as Bayesian networks [4, 64, 87] and Dempster-

Shafer’s theory of belief [24, 70] gives us the mechanism to evaluate the

outcome of systems affected by randomness or other types of probabilistic

uncertainty. Its main characteristic is its ability to update previous outcome

23


estimates by conditioning them with newly available evidence. The prob-

abilistic reasoning methods can work in an unsupervised mode in contrast

to neural networks that should be trained before it is used. Therefore, there

is no need to have a set of data to train on. The methods, along with tech-

niques like clustering that reduce the data to essentials for easier classifica-

tion, identify patterns in raw data. That is particularly important when at-

tempting to root out something like Medicare fraud and abuse from a stack

of claims. However, probabilistic reasoning methods have also drawbacks

in that the probabilistic relationships among variables should be extracted

from data set. This might require too much time if the data set is too large.

The last soft computing technique is evolutionary algorithms, especially

genetic algorithms in this dissertation. GAs, proposed by Holland [41],

give us a way to perform randomized global search in a solution space. In

this sense, a population of candidate solutions, encoded as chromosomes,

is evaluated by a fitness function in terms of its performance. The best

candidates evolve and pass some of their characteristics to their offsprings.

Soft computing, of course, has disadvantages. A soft computing tech-

nique is not appropriate for some problems. For all problems, there is the

issue that soft computing techniques find solutions that are good enough but

perhaps not the best. There is always a chance that a better solution exists

elsewhere, but it is also always true that if it exists, a good enough solution

eventually can be found. Furthermore, soft computing techniques tend to

be nonlinear ones. If the size of a problem doubles, then the size of the so-

lution and the amount of computation needed to find it become more than

doubles. Frequently the size of the solution and the amount of computation

required to ferret it out would quadruple or increase even more.

24


Behavior Controller

Evolutionary Algorithm

Fig. 3.1: Conventional soft computing framework.

3.1.2 Conventional Soft Computing Frameworks

As described earlier, two main frameworks are most commonly used

in applying soft computing techniques to the construction of intelligent

agents. The first framework is to apply each soft computing technique to

the construction of intelligent agents. This approach has advantages in its

methodological simplicity and facilities in the construction if each tech-

nique is applied to a relatively simple and tiny problem. However, it has

some drawbacks in determining the internal parameters of the techniques

because they often require expert-level knowledge and need more time and

effort.

The second framework is to combine two or more soft computing tech-

niques to relieve the difficulties of determining the internal parameters.

Especially, the combination of an evolutionary algorithm and other soft

computing technique has been focused very much as in Fig. 3.1. The be-

havior controller in Fig. 3.1 is usually implemented using soft computing

techniques, e.g., neural networks, fuzzy logic, probabilistic reasoning, etc.

Several evolutionary algorithms are available for the evolution of the be-

havior controller.

The evolutionary algorithms tune the internal parameters of the behav-

ior controller through evolution. This framework has advantages in that it

25


requires relatively less expert-level knowledge and expertise and it can find

behavior controllers that show better performance because of the global

searching capabilities of the evolutionary algorithms. Despite the good re-

sults of the combination of soft computing techniques and evolutionary al-

gorithm, this framework has at least three shortcomings.

First, the second framework cannot guarantee that the solutions have

resulted from adaptive evolution. Because, in most artificial evolutions,

adaptive evolution is the key to produce successful results, the inability to

guarantee adaptive evolution might cause serious problem.

For example, suppose you just have succeeded in getting scientifically

very important data from the last experiment of evolution, and somebody

just happen to delete your data by mistake when you are away just a few

seconds. When you realize this situation, you might think that that is ok

and just one more experiment will suffice to get the same results. However,

such a happy situation (successful evolution) would not occur that often

unless evolution is adaptive.

Adaptive Evolution

High Evolvability

Low Evolvability

Non- Adaptive Evolution

Good Solution

Bad Solution

Desirable Evolutionary Causes

and Effects

High probability

Low probability

Fig. 3.2: Possible evolutionary pathways.

26


In general, evolutionary phenomena contain not only adaptive evolu-

tion but also others such as chances and necessity. Therefore, good so-

lutions do not necessarily mean that the evolution has been adaptive (see

Fig. 3.2). In other words, if the solutions are not from adaptive evolution

but from other evolutionary phenomena, then no good solution from follow-

up experiments may be obtained even in the same experimental conditions.

Therefore, proving or showing that the solution is the result of adaptive

evolution has great importance in the justification of the experiment and

the evolution.

Second, the second framework cannot show evolutionary pathways to

the evolved solutions. Here, evolutionary pathways means that what, when,

how, and why some genes have successfully evolved into the solutions

while the others not.

Lastly, the second framework lacks methods for analyzing the obtained

solutions. Evolution necessarily causes the changes of simple structures

into complex ones that may be viewed as emergent behaviors or emergent

properties. However, the second framework cannot analyze these aspects.

3.2 Soft Computing Framework for Intelligent

Agents

In this section, a soft computing framework for intelligent agents is

proposed. Detailed description of the components of the framework will be

presented in the following sections. Though conventional frameworks can

evolve intelligent agents of good performance, they lack some analytical

27


Agent Construction

Rule-Based System

Evolutionary Algorithms

Hardware Agent

Software Agent

Evolution Analysis

Evolutionary Activity Statistics

Schema Anaysis

Observational Emergence

Analysis of Evolution

Adaptive Evolution

Adaptive Behavior Emergence

Evolutionary Pathways

to Solutions

Research goals How to achieve the goals What we can get or show

Behavior Analysis

Fig. 3.3: Proposed soft computing framework for agents.

points.

This framework augments the conventional framework of the combi-

nation of soft computing techniques and genetic algorithms by rule-based

systems and several analysis methods as in Fig. 3.3. Rule-based systems

are used for behavior controllers because they are easy to analyze when

combined with the augmented analysis methods, and understandable to hu-

mans. As a matter of course, evolutionary algorithms tune the rules in the

behavior controllers. Especially, an emphasis is put on the evolution and be-

28


havior analysis to overcome the shortcomings of the previous framework.

The analysis part in Fig. 3.3 is adopted to overcome the shortcomings of

the conventional frameworks as in the followings:

1. Evolution Analysis0 Quantification of Adaptive Evolution

The conventional frameworks cannot convince that the evolu-

tion of intelligent agents has been adaptive because several evo-

lutionary phenomena exist including adaptation. Therefore, there

are needs for certain methods for measuring or observing adap-

tive evolution. Furthermore, they should be quantitative to be

used as objective measures. In the proposed framework, evolu-

tionary activity statistics measures the adaptability of evolution

objectively and quantitatively.0 Identification of Adaptive Genes

Identifying adaptive genes is important in that those genes are

clues to uncover the secret of the evolutionary construction path-

ways to solutions, and to understand what behavioral properties

are adaptive in the problems.0 Evolutionary Pathways to Solutions

In the conventional frameworks, serious analysis on the evolu-

tionary pathways to solutions has not been done while much at-

tention has been paid to the construction of behavior controllers

of good performance. However, it is necessary to analyze the

construction pathways to solutions in order to show that the so-

29


lutions are from adaptive evolution rather than from other evo-

lutionary phenomena.

2. Behavior Analysis

Evolution is said to generate complex structures from simple struc-

tures. A complex structure is emergent in that it cannot be expected

with the simple structures that compose the complex structure. De-

cisions on whether certain behaviors are emergent should be made

relying on formal definition of emergence. We adopt the notion of

“observational emergence” [3] to judge whether an agent’s behavior

is emergent.

3.3 Rule-based Systems

Rule-based systems, especially fuzzy logic, are composed of rules based

on which the systems make decisions on which behavior to take. The rules

take the form of “IF condition THEN action.” Usually, the rules can be rep-

resented as linguistic expressions such that importing expert’s knowledge

into the systems is very easy compared to other soft computing techniques.

Although rule-based systems have advantages in that expert knowledge

can be easily represented and, therefore, easily understood by humans, it

does not have adaptation or learning features, since it lacks the mechanism

to extract knowledge from existing data.

We adopt rule-based systems as the controller for two reasons: First,

expert knowledge can be easily incorporated into the systems and repre-

sented in verbal forms. This implies that it is pretty much easier to analyze

30


a is A 1 b is B 1

a is A 2 b is B 2

a is A n b is B n

Aggregator Defuzzifier a b

Rule 0

Rule 1

Rule n

Crisp Fuzzy

Fuzzy

Fuzzy

Fuzzy

...

Fig. 3.4: Block diagram of a fuzzy controller.

the rule-based systems. Second, even when combined with evolutionary

algorithms, the degree of facility in analytic perspective does not change

so that the systems can be easily analyzed to reveal what, how, when, and

why the rules have evolved successfully or not. These aspects of facility of

analysis match best with the analysis methods adopted in the framework to

remedy the drawbacks of the conventional frameworks.

3.3.1 Fuzzy Logic

The basic structure of a fuzzy logic controller consists of three con-

ceptual components: fuzzification of the input-output variables, a rule base

that contains a set of fuzzy rules, and a reasoning mechanism that performs

the inference procedure on the rules and given facts to derive a reasonable

output. An example of a fuzzy inference system with crisp output is in

Fig. 3.4.

31


3.3.2 Fuzzy Sets and Rule Base

For a clearer explanation, assume that the fuzzy logic controller uses

the eight input variables and two output variables. Let 132#�465879�;:<�;=�=;=*�?> be

input linguistic variables defined in the universe of discourse @BA to represent

the value of the 4 th variable. A fuzzy set �C2 on 1D2 is defined as follows:�E2F5HGJILK�MFNFO$��1,2�QPR1,2#= (3.1)

The integration sign in (3.1) stands for the union of ��132#��MFNFO$��1,2� pairs,

and they do not indicate summation or integration. Similarly, “/” is only

a marker and does not imply division. In the same way, let S<2T��4U5V79�;:be output linguistic variables defined in the universe of discourse @XW to

represent the speed of the 4 th output variable, and a fuzzy set YZ2 on SL2 is

defined as follows: Y�2F5 G IL[�M]\?O$��S^2�QPLSL2 (3.2)

where the parameters _^�`��'^�?a^b with �dce'+cfa determine the g coordinates

of the three corners of the underlying triangular membership function. Gen-

erally, a human expert can construct the rule base of an FLC based on their

experience. However, when it comes to more complex work, it is difficult to

tune the rule base with only human expertise. To overcome this difficulty,

a genetic algorithm is used as an automatic way to optimize the rule base.

The rule base for the FLC consists of the rules of IF-THEN statements as

follows:

R h : IF ( 1Dhi5j� hh ) and klklk and ( 1 / 5j� h/ )

THEN ( SLhm5nY hh ) and ( S<�o5nY h� )

32


...

R � : IF ( 1Dhi5j� �h ) and klklk and ( 1 / 5n� �/ )THEN ( SLhm5nY �h ) and ( S<�o5nY �� )

...

R prq9� : IF ( 1Dhi5j� prq9�h ) and klklk and ( 1 / 58� psq9�/ )

THEN ( SLhm5nY prq9�h ) and ( S<�t58Y prq9�� ).

where R � denotes the u th rule. Also, � �2 and Y �2 , uv5n79�;:<��=;=;=��?wyxe: , are

the fuzzy sets defined on 1D2 and S^2 in the universe of discourse @6A and @zW ,respectively. Here, w denotes the number of the rules in FLC. The u th rule

can be represented as a fuzzy relation defined by

R �E{`�T� �h�| � ��C| � ��| � � �| � �} | � �% | � �~ | � �/ B� �#Y �h ��Y �� (3.3)

where � denotes fuzzy relation. The rule base can be represented as union

33


as follows:

R 5�� psq9��;�Jh R �F�5 � psq9��;�Jh �� h | � �� | � �� | � � | � �} | � �% | � �~ | � �/ 6� �TY �h ��Y �� 5�� prq9��l�Jh�� T� �h | � �� | � �� | � � | � �} | � �%U| � �~ | � �/ 6� Y �h ��T� �h | � �� | � �� | � � | � �} | � �%�| � �~ | � �/ B� Y ��r� �(3.4)5 � psq9��;�Jh6� Y �h � psq9��;�Jh6� Y �� 5�_ � Y�h;� � Y��*b

where � consists of two sub rule bases � Y�h and � YZ� . Each rule base con-

sists of w rules, each of which can be represented by a fuzzy relation in the

product space expressed as follows:@zA | @�A | @zA | @zA | @�A | @zA | @zA | @zA | @zWR= (3.5)

This fuzzy relation can be implemented with each corresponding mem-

bership function defined as follows:M��9\�� 1Dh;�19��1D�;��1D ��1 } �1 % ��1 ~ ��1 / ��SLh*5H�6��MFN]�� 1Dh*?�;=�=;=��MFN]�� 1 / ?��M�\`�� SRh�� (3.6)

34

3. SOFT COMPUTING FRAMEWORKM��J\ �. ��1Dh;��13�*�1��1D ;��1 } ��1 % �1 ~ ��1 / ��S��58�6��M�N �� 1�h�?�;=;=�=��M�N �� 1 / ?��M]\ �. ��S��= (3.7)

3.3.3 Fuzzy Reasoning and Defuzzification

Let � �2 and � �*�2 be the fuzzy sets defined on 1�2 in the universe of dis-

course @�A , and Y �h ��Y �� Y � �h and Y � �� be the fuzzy sets defined on S�h and S<�in the universe of discourse @6W , respectively. Y � �h and Y � �� can be inferred

from � �2 �?� � �2 �?Y �h , and Y �� . The u th rule R � can be transformed into a fuzzy

relation based on Mamdani’s fuzzy implication function [57] as follows:

� Y �h �T� �h ��klklk��?� �/ ��Y �h B5�� h | klklk | � �/ | Y �h5nG�I K�� I K�� IR[�M�N]�� 1Dh*F��klklk^�dM�N]�� 1 / F��M�\�� SLh�QP3��1Dh;�;klklk��1 / ��SRh�(3.8)

� Y �� T� �h �;klklk�� / ��Y �� B5��T� �h�| klklk | � �/ | Y ��5 G ILK �� ILK � IL[FM�N �� 1�h�F� klklkl�¡MFN �� 1 / F�¡M]\ �. ��S��QP9��1Dh;�;klklk��1 / �S<��=(3.9)

Based on Zadeh’s compositional rule of inference [91], YE¢h and Y£¢� are ex-

pressed as:Y ¢h 5¤�T� �*�h | klklk | � ��/ F¥ psq9��;�Jh �T� �h | klklk | � �/ � Y �h 5¤�T� � �h¦| klklk | � � �/ F¥ psq9��;�Jh6� Y �h (3.10)

35

3. SOFT COMPUTING FRAMEWORKY ¢� 5¤�T� �*�h | klklk | � ��/ F¥ psq9��;�Jh �T� �h | klklk | � �/ � Y �� 5¤�T� � �h¦| klklk | � � �/ F¥ psq9��;�Jh6� Y �� (3.11)

where ¥ denotes the maximum-minimum composition. The resulting YE¢hand Y(¢� are expressed as follows:M \ �� 5 psq9��l�Jh�§ A ��¨ �� ¨ A ��© M N]� �� 1Dh*Q�dklklk?�BM N]� �� 1 / #ª«� © M�N]�� 1�h�F�¬klklk^��MFN]�� 1 / F�¡M]\�� SLh�#ª5 psq9��l�Jh § A ��¨ �� ¨ A �6 © M N � �� 1�h�F� klklkl�¡M N � �� 1 / F��M�N �� 1Dh*F� klklk;�¡M�N �� 1 / ªZ® �dM]\ �� SRh�5 psq9��l�Jh § A � © M N]� �� 1Dh*��M�N]�� 1Dh*TªZ®¯ °�± ²³ � ��klklk^� § A �´© M N]� �� 1 / ��dM�N]�� 1 / #ª�®¯ °�± ²³ � �BM]\`�� SLh*

5 psq9��l�Jh �¶µ6h6��klklkl�·µ / ¯ °�± ²¸?¹»º ¼?½z¾À¿À¹ÂÁ¶¼?½#¿»Ã �BM]\ �� SRh��=(3.12)

Similarly, M \ �. is defined as:M \ �. 5 prq9��l�Jh ��µ�h6��klklk;�dµ / ¯ °�± ²¸?¹Àº ¼?½�¾»¿Â¹ÀÁ¶¼?½#¿ÀÃ �BM]\ �. ��S<�� (3.13)

where � denotes the minimum operation, and µB2 is the maxima of the mem-

bership functions of � �2´Ä � �*�2 .

Defuzzification refers to the way ÅS�h and ÅS�� are extracted from a fuzzy set

as representative values. Among many defuzzification methods [57][89],

36


the center of gravity method is most frequently used as in (3.14, 3.15).ÅSLht5yÆ W � MZW � ��SRh�#SRh?1DSLhÆ W � MFW � ��SRh�Ç1DSRh (3.14)

ÅS��o5 Æ W. MFW�.��S��ÇS��1DS<�Æ W�. MFW.��S<��#1�S�� = (3.15)

3.4 Evolutionary Algorithms

The evolutionary algorithms cover several variations such as evolution-

ary strategies, evolutionary programming, genetic programming, and ge-

netic algorithms. Evolutionary algorithms are to evolve the behavior con-

trollers of agents. As the controllers are usually implemented using soft

computing techniques, the role of the algorithms is to tune the internal pa-

rameters of the techniques.

Although the origins of evolutionary algorithms can be traced back to

the late 1950’s [14, 15, 30], the field remained relatively unknown to the

broader scientific community for almost three decades until the fundamen-

tal work of Fogel [28], Holland [39], and Rechenberg [65].

The most significant advantage of using evolutionary algorithms lies in

the gain of flexibility and adaptability to the task, in combination with ro-

bust performance and global search characteristics. In fact, evolutionary

algorithms should be understood as a general adaptable concept for prob-

lem solving, especially well suited for solving difficult optimization, ma-

chine learning, and searching problems, rather than a set of ready-to-use

algorithms.

37


3.4.1 Genetic Algorithms

Genetic algorithms have had a great measure of success in search and

optimization problems. The reason for a great part of their success is their

ability to exploit the information accumulated about an initially unknown

search space in order to bias subsequent searches into useful subspaces, i.e.,

their adaptation. This is their key feature, particularly in large, complex,

and poorly understood search spaces, where classical search tools (enumer-

ative, heuristics, ...) are inappropriate, offering a valid approach to prob-

lems requiring efficient and effective search techniques [81].

GAs are general-purpose search algorithms which use principles in-

spired by natural genetic populations to evolve solutions to problems [35,

41]. The basic idea is to maintain a population of chromosomes, which rep-

resent candidate solutions to the concrete problem that evolves over time

through a process of competition and controlled variation. Each chromo-

some in the population has an associated fitness to determine which chro-

mosomes are used to form new ones in the competition process, which is

called selection. The new ones are created using genetic operators such as

crossover and mutation.

Although there are many possible variants of the basic GA, the funda-

mental underlying mechanism consists of three operations:

1. Evaluation of individual fitness

2. Formation of a gene pool (intermediate population) through selection

mechanism

3. Recombination through crossover and mutation operators

38


The basic principles of GAs were first laid down rigorously by Hol-

land [41], and well described in many books such as [35]. It is generally

accepted that the application of a GA to solve a problem must take into

account the following five components:

1. A genetic representation of solutions to the problem

2. A way to create an initial population of solution

3. An evaluation function that gives the fitness of each chromosome

4. Genetic operators that alter the genetic composition of offspring dur-

ing reproduction

5. Values for the parameters that the GA uses (population size, proba-

bilities of applying genetic operators, etc.)

GAs may deal successfully with a wide range of problem areas. There

are four main reasons for the success. GAs can solve hard problems quickly

and reliable. GAs are easy to interface to existing simulations and models.

GAs are extensible. GAs are easy to hybridize. All these reasons may be

summed up in only one: GAs are robust. GAs are more powerful in difficult

environments where the space is usually large, discontinuous, complex, and

poorly understood. They are not guaranteed to find the global optimum

solution to a problem, but they are generally good at finding acceptably

good solutions to problems acceptably quickly. These reasons have been

behind the fact that, during the last decade, GA applications have grown

enormously in many fields such engineering and computer science [86],

machine learning [36, 52, 85], pattern recognition [63], robotics [31, 44,

54], and fuzzy systems [11, 54, 60].

39


3.4.2 Genetic Operations

A genetic algorithm assigns each candidate solution (usually encoded

as bit strings) in a population with an associated fitness value measuring

the candidate’s survivability, and this process is called evaluation. In ad-

dition, the GA evolves this population of individual candidates into a new

population using three operators: selection, crossover, and mutation. A GA

does its search through an iterative process. The process of one generation

involved with selection, crossover, and mutation is called one cycle of it-

eration. Selection probabilistically chooses better candidate solutions for

a new generation. Crossover and mutation manipulate candidate solutions

to generate new individuals for selection to process again. The following

paragraphs discuss in detail how these three operators work.

Selection probabilistically chooses individuals according to their fit-

nesses. The higher the fitness, the more likely an individual is selected.

In natural systems, an organism, or phenotype, formed by a genetic pack-

age (genotype) participates in the process of “survival of the fittest.” Ad-

vantageous genotypes are selected based on their survivability. Similarly,

in an artificial genetic system, advantageous bit strings, the encoded ver-

sions of candidate solutions in the solution space, are selected based on the

fitnesses of their associated genotypes. Goldberg’s book [35] mentions dif-

ferent selection methods such as roulette wheel selection, elitist selection,

and scaling. Roulette wheel selection is similar to spinning a roulette wheel

in which the size of the roulette wheel slot for each individual in the pop-

ulation is allocated in proportion to its fitness. Thus, the individuals with

higher fitness have better chance of being selected relative to the less fit

40


1 0 0 1 1 0 1 1 0 1 1 1 1 0 1 1 0 1 1 0

Crossover Point Crossover Point

1 1 1 1 0 0 0 1 1 0 1 1 0 1 1 0 1 1 0 1

Parents

Offspring

Fig. 3.5: Single point crossover.

individuals.

However, roulette wheel selection does not guarantee that high fitness

candidate solutions propagate to the next generation. This selection method

is less effective because of its high stochastic error. The elitist selection

method guarantees a certain number of high fitness individuals will prop-

agate to the next generation, and decreases stochastic errors. Scaling uses

a scale factor to keep the competition of population members at a certain

level.

Crossover is the mating process allowing for information exchange.

Fig. 3.5 shows how one-point crossover creates two new individuals through

swapping all the bits beyond the randomly chosen crossover point.

The crossover probability controls how often the crossover operator is

applied. The higher the crossover rate, the more quickly new candidate

solutions are introduced into the population. If the crossover probability

is too high, highly fit individuals are discarded faster than selection can

produce improvements. On the other hand, if the crossover rate is too low,

the search might stagnate for lack of exploration. Usually, the crossover

probability is chosen to lie between 0.5 and 1.0.

41


1 1 1 1 1 0 1 1 0 1 Offspring

1 1 1 1 0 0 1 1 0 1 Mutated Offspring

Mutation Point

Fig. 3.6: A single mutation.

The mutation operator randomly chooses a mutation point with a low

probability, and flips the bit at that point. Fig. 3.6 shows an example of

a chromosome with one bit being mutated. Mutation expands the scope of

exploration. Overall, mutation plays a secondary role in genetic algorithms.

Fig. 3.7 shows the general procedure of a GA. A genetic algorithm is

run independently several times using different random seeds to reduce the

dependency of performance on random seeds. Thus, the average fitness

over several independent runs represents performance more realistically.

Genetic algorithms dynamically balance exploration with exploitation

through selection, crossover, and mutation. A high probability of crossover

and mutation increases exploration. Different selection methods have dif-

ferent degrees of exploitation. Elitist selection exploits more in the highly

fit areas over roulette wheel selection because it preserves high fitness in-

dividuals in the next generation. Thus for roulette wheel selection, smaller

mutation rates should be used because of high stochastic error. For elitist

selection, relatively higher crossover and mutation rates should be used.

The balance of exploration and exploitation needs to be considered care-

fully for two reasons. First, the speed and efficiency of a GA may suffer

42


Initialize New Population

Evaluate Population

Apply Crossover and Mutation

Select and Reproduce Population

Stop

Terminal Condition

yes

no

Fig. 3.7: General procedure of GA.

43


if too much exploration is conducted in unpromising regions. Second, too

much exploitation in unpromising areas may cause a GA to lose critical

information on correct solutions and lead it to converge on a local opti-

mum. Choosing appropriate crossover and mutation rates is important in

obtaining good performance for genetic algorithms.

The fact that GAs work on a population of individuals, rather than a

single individual, is an essential part of the algorithm. The population size

determines how many initial candidate solutions are available for GAs and

affects performance and efficiency. A small population size may provide

insufficient sample size. A larger population size provides more candidate

solutions; however, it requires more evaluations per generation and results

in slower convergence.

3.4.3 Learning with GA

Although GAs are not learning algorithms, they may offer a powerful

and domain-independent search method for a variety of learning tasks. In

fact, there has been a good deal of interest in using GAs for machine learn-

ing problems [35, 52, 85].

Two main alternative approaches, in which GAs have been applied to

learning processes, have been proposed, the Michigan [40] and the Pitts-

burgh [74]. In the former, the chromosomes correspond to classifier rules

evolved as a whole, whereas in the Pittsburgh approach, each chromosome

encodes a complete set of classifiers.

Michigan approach. The chromosomes are individual rules and the

44


entire population represents a rule set. The collection of rules is modified

over time via interaction with the environment. This model maintains the

population of classifiers with credit assignment, rule discovery, and genetic

operations applied at the level of the individual rule. A genetic learning

process based on the Michigan approach receives the name of Classifier

System or Learning Classifier System [13, 42].

Pittsburgh approach. Each chromosome encodes a whole rule sets.

Crossover serves to provide a new combination of rules and mutation pro-

vides new rules. In some cases, variable-length rule bases are used, em-

ploying modified genetic operators for dealing with these variable-length

and position independent genomes. This model was initially proposed by

Smith in 1980 [74].

The Michigan approach will prove to be the most useful in an on-line

process. It is more flexible to handle incremental-mode learning (training

instances arrive over time) and dynamically changing domains, whereas

the Pittsburgh approach seems to be better suited to batch-mode learning,

where all training instances are available before learning is initiated, and

for static domains.

The major problem in the Michigan approach is that of resolving the

conflict between the individual and collective interests of classifiers within

the system. The ultimate aim of a learning classifier system is to evolve

a set of co-adapted rules that act together in solving some problems. In

a Michigan style system, with selection and replacement at the level of

the individual rule, rules that cooperate to effect good actions and receive

45


payoff also compete with each other under the action of the GA.

This conflict between individual and collective interests of individual

classifiers does not arise with Pittsburgh-style classifier systems, since re-

productive competition occurs between complete rule sets rather than in-

dividual rules. However, maintenance and evaluation of a population of

complete rule-sets in Pittsburgh-style systems can often lead to a much

greater computational burden (in terms of both memory and processing

time). Therefore, problems with the Pittsburgh approach have proven to

be, at least, equally as challenging. Although the approach avoids the prob-

lem of explicit competition between classifiers, large amounts of computing

resources are required to evaluate a complete population of rule-sets.

3.5 Analysis Methods

This section describes the methods for the analysis of evolution and

emergence of behaviors of agents. In section 3.5.1, we discuss evolution-

ary activity statistics used to quantitatively measure the adaptiveness of an

evolution and to find adaptive genes during the evolution. Section 3.5.2

presents the definition of observational emergence used to analyze the be-

haviors of agents to determine whether their behaviors are emergent.

3.5.1 Evolutionary Activity Statistics

The goal of evolutionary systems is to find the best solutions that can

solve the problem at hand. Although these solutions have been proven to

work well in applications of many areas, no proof has been done that the

46


best solution is the result of adaptation to the given problems. Actually, they

might be the results of other genetic phenomena like chance and necessity.

The capacity to produce good solutions via evolution, evolvability, can

be defined in various ways as “genome’s ability to produce adaptive vari-

ants when acted on by the genetic system” by Wagner and Altenberg [82],

“capacity to generate heritable phenotypic variation” by Kirshner and Ger-

hart [50], or “capacity to create new adaptations, and especially new kinds

of adaptations, through the evolutionary process” by Bedau [8]. To obtain

good solutions robustly, the evolvability should be maintained during the

evolutionary process. Under this condition, good candidate solutions can

contribute to the formation of the solutions.

Adaptability shows different characteristics compared to other genetic

phenomena. Adaptive components in evolutionary systems generate inno-

vative functional structures continuously and these structures persist in the

systems because they have high adaptive behavioral characteristics. There-

fore, if a system evolves adaptively, it means that the systems continuously

generate new functional structures, which persist through the evolutionary

process.

Based on this idea, Bedau proposed a new measure, evolutionary ac-

tivity [6]. He tried to quantify the degree to which a system exhibits the

continual spontaneous generation of adaptive forms. Specifically, it mea-

sures the degree to which new genetic combinations are persistently used in

a population. This quantity is an objective, empirical measure of the level

of evolutionary activity in an artificial or natural system.

Evolutionary activity is to identify innovations that make a difference.

Generally, an innovation can be seen if it persists and continues to be used.

47


A counter is attached to each component to measure the evolutionary ac-

tivity. A component is a unit that can play a significant role in a given

problem. A component, therefore, can be at different levels depending on

the problems at hand. For example, in a system where each genotype is

contending for space over time, a genotype’s capability or adaptability can

be measured by the amount of space it occupies. In this case, components

are the genotypes and counters are attached to them. Individual alleles or

taxonomic families can also be components depending the problems [6].

A counter, �D2#�� , of the 4 th component at time � is attached to each

component of a system. A component’s activity changes over time as in

(3.16). �,2Ç��B5nÈ É�ÊDË`ÌE2Ç�#Í� (3.16)

where ÌE2$�#Í� is the activity increment for component 4 at time Í . Various

activity increment functions can be used, depending on the nature of the

components and the purposes at hand [6]. One could increment a compo-

nent’s activity with its concentration in the system or the extent to which

it is used or expressed [6] in contexts in which either of those properties

tended to be correlated with adaptive value.

Various statistics can be defined based on the components in a system

and their activity counters. Diversity, � �� , is the number of components

at time � defined as in refequ:diversity).�v��65HÎ�_^4s{,�,2Ç��oÏÐ73b (3.17)

48


where Î denotes set cardinality.

The number of components with activity � at time � is the component

activity distribution, �U��J , defined as in refequ:cad).��JB5 È 28Ñ ��+xÒ�D2#�� (3.18)

where Ñ ��xÓ�,2Ç�� is the Dirac delta function, equal to one if �C58��2Ç�� and

zero otherwise. To measure the continual adaptive success of the compo-

nents in the system at a give time, total cumulative evolutionary activity or

total activity in short, �� , is defined as follows:��B5 È 2 �,2Ç�� GÕÔh ��JÇ1D�`= (3.19)

Dividing the cumulative evolutionary activity by diversity � �� gives the

cumulative activity per component, mean cumulative evolutionary activity

or mean activity in short, as follows:��B5 �� (3.20)

Activity statistics most clearly reflect a system’s adaptive evolutionary dy-

namics after they have been normalized by a “neutral” model [66], to screen

off the contribution of non-adaptive or maladaptive genotypes. This nor-

malization can be used for several measures.

Adaptive innovations correspond to new components flowing into the

system and proving their adaptive value through their persistent activity.

49


Let ��h and �3� define a strip through the component activity distribution

function, ��J , such that activity values � in the range �9hEÖ��ÓÖ¤�9� are

among the lowest activity values that can be interpreted as evidence that

a component has positive adaptive significance. Then, new evolutionary

activity or new activity in short, �´�*�T�� , is the reflection of the evolutionary

adaptive innovations by summing the activity per component with values

between ��h and �9� as in (3.21).�«��T�6��65 :� ��¤× .È× � × � ��J� :� �� G × .× � ��JÇ1D�(3.21)

3.5.2 Observational Emergence

Emergence has played a major role in the description and discussion

of natural and artificial life, and complex systems. In an intuitive point of

view, it is used as a name for creation of new structures and properties as the

old philosophical statement, “the whole is more than the sum of its parts.”

For more formal definition of emergence, Baas’ notion of emergence is

used [3].

The definition of emergence starts with a general notion of structures as

primitive objects or entities. A structure may be of an abstract or physical

nature, e.g., systems, organizations, organisms, machines, concepts, etc.

Furthermore, assume that some kind of observational mechanism (or family

of such) exists to evaluate, observe, and describe the structure. This could

be an internal mechanism of the system as well as an external one.

50


To give a general procedure for how to construct a new structure from a

family of old ones, let us start out with a family of structures,_R-Z2�bD��4sØÚÙ (some index set, finite or infinite) = (3.22)

Then observational mechanisms, &+'*) , are applied to obtain properties of

the structures, -Z2T�Û&+'*),�#-Z2�?= (3.23)

Next the -Z2 ’s are put to a family of interactions, Ü,uZ� , using the properties

registered under the observation. Hence, a new kind of structure is obtained

as follows: - 5 � �#-Z2T�?&+'*),�#-Z2�?�?Ü,u��?��4sØÚÙ (3.24)

where � stands for the result of the construction process. Here, - is a

second-order structure as opposed to the -]2 ’s which are first-order struc-

tures. The interactions may be caused by the structures themselves or im-

posed by external factors. At each level of construction, new properties or

new behaviors may emerge, giving room for new interactions, and hence

each level is necessary in order to get the last level’s properties. Therefore,

the w th order structure is defined as follows:- p 5 � �#- prq9�2ÞÝ`ß . �?&('�) psq9� �?Ü,uZ� psq9� ?�à4Tprq9�mØáÙ,psq9� (3.25)

where w means the w th order and 4#prq9� means the 4 th structure of thewâxe: th order. From this, we can introduce the definition of emergence as

51


follows:ãis an emergent property of - �äUå ã ØÓ&+'*) � �T- � �� but

ãyæØ¬&('*) � �#- �Lq9�2 � ß . for all 4��Lq9�*= (3.26)

Emergence is deducible or computable if there is a deductional or com-

putational process or theory � such that

ã Ø &+'*) � �T- � can be determined

by � from �T- �Lq9�2 � ß . �?&+'*) �Rq9� ��ÜDu�� Rq9� , and observational if

ãis an emergent

property but cannot be deduced as in deducible.

3.5.3 Schema Analysis

For genetic algorithms, a significant concept deals with a string tem-

plate known as a schema. A schema is a similarity template representing a

subset of strings with similarity at certain string positions. For example, the

schema 1##0 represents all the bit strings with ‘1’ in the first position and

‘0’ at the fourth position. The ‘#’ notation represents “do-not-care” bit and

can be either ‘1ï or ‘0ï. The bit strings 1000, 1010, 1100, and 1110 are all

the instances of the schema 1##0. Schemas have two properties: Schema

order and schema defining length. Schema order is the number of fixed

positions (for example, the order of the schema of #100# is 3). Schema

defining length describes the distance between the first and the last specific

string positions (for example, the schema 1#0#1 has a defining length of

4). Crossover and mutation disrupt schemas. The number of copies of a

particular schema ç that can survive to the next generation ��H: is given

52


by: è �Tç��]�f:Roé è �Tç��zk �6�Tç¬Å� ê :ëx ã � Ñ �Tç�ì xe: xá&��Tç¬ ã �Xí�= (3.27)

The fitness of a schema is the average fitness of all bit strings repre-

sented by the schema.

ã � and

ã � stand for the crossover probability and

mutation rate.ì

is the length of the bit string, thusì xn: is the number of

possible places crossover can occur. Ñ ��ç� and &��ç� represent the schema

defining length and the schema order respectively. This formula shows that

short, low order, above-average schema receive exponentially increasing

trials in the subsequent generations [35]. This is known as the schema the-

orem and is the fundamental theorem of genetic algorithms [41].

53

CHAPTER 4

APPLICATION TO A

HARDWARE AGENT

In this chapter, we apply the framework to the construction of a behav-

ior controller for a hardware agent. Fig. 4.1 shows the framework applied

to the construction of the hardware agent. The hardware agent called Khep-

era is a kind of behavior-based mobile robot. Eight proximity sensors are

used as input to the behavior controller and two control values are produced

from the controller for the two motors of the robot. Fuzzy logic is applied

for the behavior controller of the robot. As the framework explains, a ge-

netic algorithm is used to tune the internal parameters of the fuzzy behavior

controller. The evolved controller and the evolution are analyzed based on

the analysis methods.

54

4. APPLICATION TO A HARDWARE AGENT

Agent Construction

Rule-Based System

Fuzzy Logic Controller


Genetic Algorithm

Evolution Analysis

Evolutionary Activity

Schema Anaysis



Adaptive Evolution



to Solutions


Behavior Analysis

Fig. 4.1: A framework for hardware agents.

55


4.1 Introduction

Fuzzy logic controllers [2, 11, 31, 37, 45, 48, 54, 69, 80, 90] have been

widely used for behavior-based robots like Khepera because they can easily

transform linguistic information and expert knowledge into control signals.

While fuzzy logic control has many advantages over traditional methods,

it has also some drawbacks at the design stage in that it is difficult to de-

termine the optimal parameters. Therefore, many researchers have applied

evolutionary algorithms to the construction of FLCs in order to automate

the procedure of determining the parameters [37, 54, 55, 69].

Many previous works on evolutionary fuzzy logic controller have shown

successful results for the problems at hand. However, most of previous

works do not provide any in-depth analysis on the role of evolutionary al-

gorithms in finding the rules contained in the optimal controller.

We aim to show the role of evolutionary computation in finding the rules

that are successful for solving problems by a measure of evolutionary ac-

tivity proposed by Bedau and Packard [6, 7]. Although adaptive evolution

is known to make complex functional structures [41, 73], it is generally dif-

ficult to differentiate adaptive change from other evolutionary phenomena

such as chance and necessity. Measurement of the evolutionary activity of

a system is an objective, quantitative method for the analysis of the evolu-

tionary pathway of the evolved fuzzy controller.

In this case study, we will give quantitative analysis that the evolved

fuzzy controller is not the result of other evolutionary phenomena but that

of adaptability of the rules that constitute the fuzzy logic controller, by ap-

plying the evolutionary statistics and observing the changes of schemata of

56


(a) Khepera robot.

IR proximity sensors

0

1

2 3

4

5

6 7

Motor 1 Motor 0

(b) Position of theeight IR sensors andtwo motors.

Fig. 4.2: (a). Khepera mobile robot (b). IR sensors and motors [46].

the evolved rules. We screen off the effect of non-adaptive evolutionary

forces by comparing the evolutionary dynamics observed in the evolution

of fuzzy logic controller, “Fuzzy Model,” with those observed in analogous

evolutionary systems, “Neutral Shadow,” in which adaptive evolution can-

not happen. We will also analyze the evolved control mechanism to show

that the behaviors of the mobile robot emerge from the interactions of sev-

eral primitive rules.

4.2 Hardware Agent: KHEPERA

Khepera (see Fig. 4.2) was originally designed for research and teach-

ing in the framework of a Swiss Research Priority Program [46]. It allows

confrontation to the real world of algorithms developed in simulation for

trajectory execution, obstacle avoidance, pre-processing of sensory infor-

mation, and hypothesis test on behavior processing.

57


1

2

3

4

5

6

5

7 6

7

8

Top View Side View Bottom View

Fig. 4.3: Position of some parts of the robot [46]. À LEDs. Á Serial line(S) connector. Â Reset button. Ã Jumpers for the running mode selection.Ä Infra-Red proximity sensors. Å Battery recharge connector. Æ ON-OFFbattery switch. Ç Second reset button.

Khepera robot has two wheels. A DC motor coupled with the wheel

through a 25:1 reduction gear moves every wheel. An incremental encoder

placed on the motor axis gives 24 pulses per revolution of the motor. This

allows a resolution of 600 pulses per revolution of the wheel that corre-

sponds to 12 pulses per millimeter of path of the robot. The motor con-

troller can be used in two control modes: speed and position modes. The

active control mode is set according to the kind of command received. If

the controller receives a position control command, the control mode is

automatically switched to the position mode. Used in speed mode, the con-

troller has as input a speed value of the wheels, and controls the motor to

keep this wheel speed. The speed modification is made as quick as pos-

sible, in an abrupt way. No limitation in acceleration is considered in this

mode.0 The normal ambient light. This measure is made using only the re-

ceiver part of the device, without emitting light with the emitter. A

new measurement is made every 20ms. During the 20ms, the sensors

are read in a sequential way every 2.5ms. The value returned at a given

58


time is the result of the last measurement made.

4.3 Rule-Based System: Fuzzy Logic Controller

The basic structure of a fuzzy logic controller consists of three con-

ceptual components: fuzzification of the input-output variables, a rule base

that contains a set of fuzzy rules, and a reasoning mechanism that performs

the inference procedure on the rules and given facts to derive a reasonable

output.

Navigation of a mobile robot in uncertain environments requires avoid-

ing obstacles. In order to reach a destination point without colliding into

obstacles, a mobile robot should take sensory information about obstacles

into account. Our FLC uses the sensory information of eight proximity

sensors as inputs and controls the speed of the two motors on Khepera.

The input linguistic variable and output linguistic variable are expressed

by linguistic values (VF, F, M, C, VC) and (BF, B, S, F, FF) respectively.

The linguistic terms have the following meanings:

Input Variable

VF : Very Far

F : Far

M : Medium

C : Close

VC : Very Close

Output Variable

BF : Backward Fast

B : Backward

S : Stop

F : Forward

FF : Forward Fast

The membership functions are all in a triangular form defined by equa-

59


tion (4.1).

triangle ��g]��`��'^�?a;65 è �Dg¬î è 4TuÚî gUxÒ�'oxÒ� � atxÚgatxá']ï ��7 ï (4.1)

where the parameters _^�`��'^�?a^b with �dce'+cfa determine the g coordinates

of the three corners of the underlying triangular membership function. To

reduce the computational complexity, some restrictions are applied to each

membership function as in the following equation.MFAQO ¨ \`ðB��1,2�B5 triangle �T73�?79�?a ðAQO MFAQO ¨ ðB��1,2�B5 triangle �T73�?a ðAQO �?a�ñA�O MFAQO ¨ ñ ��1,2�B5 triangle �Ta ðA�O �?a�ñAQO ��a*òAQO MFAQO ¨ ò ��1,2�B5 triangle �Ta�ñA�O �?a�òA�O ��:l7,ó�ô�MFAQO ¨ \ ò ��1,2�B5 triangle �Ta òA�O �;:;7,ó�ô9�;:;7,ó�ô�MFWÇO ¨ õ ðX��S^2�B5 triangle ��x(:l79��x+:l73�?a õWÇO MFWÇO ¨ õ ��S^2�B5 triangle ��x(:l79��a õWÇO �?a�öWÇO MFWÇO ¨ ö ��S^2�B5 triangle �Ta õWÇO �?a*öW#O �?a ðW#O MZWÇO ¨ ðB��S^2�B5 triangle �Ta öWÇO �?a ðW#O ��E:l7�MZWÇO ¨ ð�ðB��S^2�B5 triangle �Ta ðWÇO ��E:l79��E:l7�(4.2)

where 7¦c÷a ðAQO c÷a ñAQO c÷a òA�O c :l7,óRô and x(:l7�cøa õW#O cøa öWÇO cVa ðWÇO c�E:l7 . a�AQO is one of a ÉAQO �?Íj5ù79�;=;=;=*��ó�7 , evenly distributed in the universe

of discourse, i.e., the range of input variables, while a*WÇO is one of a ÉWÇO ��Í¬579�;=;=�=��ó�7 , evenly distributed in the universe of discourse, i.e., the range of

output variables.

60


8 INPUT 2 OUTPUT 10 RULES

Fig. 4.4: Gene representation of an individual.

To control the actions of mobile robot, Mamdani’s fuzzy implication

function [57] and Zadeh’s compositional rule of inference are used to in-

fer the control values for the two motors from the eight input variables or

proximity sensors. Center of gravity method is used to extract control val-

ues from a fuzzy set as representative values.

4.4 Evolution of Behavior Controller

The code of an individual representing the parameters of FLC is ap-

plied to the Khepera robot in the environment to measure the fitness. After

some period, each individual FLC is given a fitness value according to its

performance in a given domain. The individuals with higher fitness are se-

lected and genetic operations are applied to produce the next population of

individuals.

Two parameters should be determined to run GA: how to encode the

FLC parameters in gene code and how to estimate the fitness value of each

individual. For the FLC parameters, eight input variables, two output vari-

ables, and maximally ten rules are encoded as in Fig. 4.4.

Three of the five fuzzy membership functions need to be encoded as

in Fig. 4.5 because the other two membership functions have the center

positions fixed.

Each rule has eight input variables, 13h;��=;=;=��1 / , and two output variables,

61


1 2 3 4 5 6 7 8 9

Fig. 4.5: Three out of five membership functions need to be encoded.

0 0 4 0 3 1 2 1 0 4 1

d 0 d 1 d 2 d 3 d 4 d 5 d 6 d 7 v 0 v 1

2 1 0 1 3 0 1 0

1 2

variable toggle flag

rule toggle flag

1 conditional part

2 consequent part

Fig. 4.6: Encoding of a rule.SRh and S<� . Variables having the toggle bit ‘1’ participate in the conditional

part in a fuzzy rule. Output variables do not have a toggle flag because all

of them should appear in the consequent part. The last bit in Fig. 4.6 desig-

nates whether this rule participates in fuzzy inference process. Therefore,

Fig. 4.6 can be decoded as follows:ú$û ��1Dhi5nü¬þý�ÿ�� 13�o5�� û ]ý�ÿ��·��1 ~ 5��«zý�ÿ��·��1 / 58ü�� SLhm5� û þýRÿ��·��S��r5 ûBû �=The fitness of an individual, � , with ) run time steps and u check-points

passed through is defined by

�H5�� uw �È Ë �Jh Y Ë ��:ëx � � Ë ��79= ó«� 79=��J��:mx Ü Ë (4.3)

where w is total number of checkpoints, Y Ë is average rotational speed at

62


step � , � Ë is the normalized absolute value of the difference between the

speed of the two wheels, and Ü Ë represents the normalized value of the sen-

sor which presents the highest level of activation.

A genetic algorithm is used for the evolution of fuzzy rules. Fifty indi-

viduals are randomly initialized at the beginning of evolution. Maximum

generation is 900. Population is overlapped by 50 percentages with eli-

tist strategy. New individuals replace individuals with relatively low fitness

values. Crossover rate is 0.5, and mutation rate is 0.01.

A neutral shadow of the fuzzy model is also experimented to screen off

the contribution of non-adaptive or maladaptive genotypes and to measure

the excess evolutionary activities of genotypes [66]. All the parameters

of the neutral shadow model are the same with the fuzzy model except

that the selections occur randomly regardless of fitness. The evolutionary

dynamics in a neutral shadow is a neutral diffusion process in genotype

space. Genotypes arise and go extinct, and their concentrations change over

time, but the genotype dynamics are at best weakly linked to adaptation. All

the selections in a shadow model are at random so that no genotype has any

adaptive significance.

As the number of whole rules in a population is constant, the number

or ratio of the rule’s genotype expresses a rule’s relative significance. Rel-

atively well-adapted rules will have a high concentration in a population,

and relatively poorly adapted rules will be correspondingly scarce. There-

fore, the evolutionary activity ��2#�� of the 4 Ë�� rule at time � is defined as

its instances integrated over the period from its origin up to � , provided it

63


Fig. 4.7: Environment.

exists: �,2$��B5 �� Æ Ëh uZ2$��Ç1,� if rule 4 exists at �7 otherwise(4.4)

where uZ2#�� is the number of rule instances of genotype 4 at � .Fig. 4.7 shows the environment where each individual is evaluated. The

environment is set up such that the robot can meet many different situations

like “left turn,” “right turn,” “narrow path,” and “open area.” Here, the goal

of the robot is to reach the goal point starting from the position (depicted

as 0 in Fig. 4.7) by moving along the solid line in Fig. 4.7.

Fig. 4.8 (a) shows fitness changes over generation of the simulation.

Best fitness shows typical punctuated evolution with four jumping points.

Average fitness mimics the changes of the best fitness after a short gen-

erational interval. The best that reaches the goal has appeared at genera-

tion 816, after four punctuated increments in fitness have occurred. The

movements of the best individual is in Fig. 4.8 (b). As can be seen, the

best individual shows behaviors like “turning left,” “turning right,” “pass-

64


0 100 200 300 400 500 600 700 800 9000

50

100

150

200

250

300

350

400

Generation

Fitn

ess

AverageBest

(a) (b)

Fig. 4.8: (a). Fitness changes. (b). Trajectory of the best individual.

ing through narrow corridor,” and “moving in wide area,” which we did not

specify directly in the gene codes.

While the best individual in Fig. 4.8 (b) has seven rules with their rule

toggle flag ‘1,’ only five of them are actually used during the simulation.

These five rules are in Table 4.1, and their associated fuzzy sets are as in

Fig. 4.9.

4.5 Evolution Analysis

4.5.1 Adaptive Evolution

In this section, we illustrate that the performance shown by the best in-

dividual results from adaptation to the given problem by utilizing the con-

cept of evolutionary activity proposed by Bedau [6]. Fig. 4.10 shows the

65


Table 4.1: Rules of the best individual.

Rule Gene Code MeaningB � 0111001102140000311 ( 1��s5 F) and ( 1D i5 F) and ( 1 % 5 VC)� ( SRhi5 F) and ( S<�r5 B)B � 0003120304010103401 ( 19�o5 M) � ( SLhi5 FF) and ( S<�r5 BF)B 0304000410130204031 ( 1 } 5 VF) and ( 1 % 5 C)� ( SRhi5 BF) and ( S<�t5 F)B } 0102041003010411441 ( 19 o5 VF) and ( 1 / 5 F)� ( SRhi5 FF) and ( S<�r5 FF)B % 0400110114000110211 ( 19�o5 F) and ( 1 } 5 VC) and ( 1 / 5 VF)� ( SRhi5 S) and ( S<�r5 B)

0 1023

VF F M C VC VF F M C VC




d 0

d 2

d 4

d 6

d 1

d 3

d 5

d 7

-10

BF B S F FF BF B S F FF

d 6 d 7

0 1023

0 1023

0 1023

0 1023

0 1023

0 1023 0 1023

+10 -10 +10

Fig. 4.9: Fuzzy sets obtained by evolution.

66


component activity distribution function. This graph depicts how evolu-

tionary activity of the rules on the ��x axis varies as a function of time on

the g]x axis. Each wave corresponds to a single rule and shows the variation

over time of that rule’s evolutionary activity.

By the definition of evolutionary activity as in (3.16), the slope of a

given rule’s activity wave at a time can be interpreted as the rule’s concen-

tration in the population at that time because the number of all the rules in

a population at a time is constant. When a new rule appears in the popula-

tion, a new wave arises from the g]x axis. As the rule’s concentration in the

population grows (or shrinks) over time, the slope of the wave increases (or

decreases). When the rule finally goes extinct, the slope of its wave falls

to zero, and the wave ends. In this way, a rule’s activity wave reflects its

changing concentration throughout its history in the population. Whenever

one rule drives another to extinction, a new wave arises as an earlier one

dies out. The dominating rules during a given epoch of evolution appear as

dominating waves.

Fig. 4.11 (a) shows the time series of diversity � . The diversity values

of neutral shadow are pretty much higher than those of the fuzzy model.

This arises from the fact that there is no selective pressure in the neutral

shadow model in contrast to the fuzzy model, where more adaptive indi-

viduals have higher possibilities of being selected, and, therefore, produce

more children and persist over time.

Fig. 4.11 (b) shows the total cumulative evolutionary activity, �£�� , and

Fig. 4.11 (c) shows mean cumulative evolutionary activity, �(�� , of the two

models. �� and �� are significantly higher in the fuzzy model than in

the neutral shadow. This means that much more adaptive rules are present

67


Fig. 4.10: Activity distribution function �U��J .in the fuzzy model than in its neutral shadow.

Fig. 4.12 illustrates the difference of the component activity distribu-

tions between the fuzzy model and its neutral shadow. The distributions

in Fig. 4.10 have been summed along the temporal dimension and then di-

vided by the total number of counts in both distributions. Therefore, the

value of each distribution at a given activity value � represents the fraction

of activity counts in each distribution that have activity � .As can be seen in Fig. 4.12, the fraction of activity counts of neu-

tral shadow at relatively lower activities is higher than that of the fuzzy

model. However, at higher activities, the fraction of activity counts of neu-

tral shadow is much lower than that of the fuzzy model. This means that

high adaptive rules are much more in the fuzzy model than in its neutral

shadow.

68


0 100 200 300 400 500 600 700 800 900150

200

250

300

Generation

Div

ersi

ty

Fuzzy ModelNeutral Shadow

(a) Diversity.

0 100 200 300 400 500 600 700 800 9000

0.5

1

1.5

2

2.5x 10

4

Generation

Tota

l Act

ivity

Fuzzy Model Neutral Shadow

(b) Total activity.

0 100 200 300 400 500 600 700 800 9000

20

40

60

80

100

120

Generation

Mea

n A

ctiv

ity


(c) Mean activity.

0 100 200 300 400 500 600 700 800 9000

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

Generation

New

Act

ivity


(d) New activity.

Fig. 4.11: (a). Diversity �v�� . (b). Total cumulative evolutionary activity�� . (c). Mean cumulative evolutionary activity �´�� . (d). Newactivity ��6�� .

69


100 101 102 103 10410−6

10−5

10−4

10−3

10−2

10−1

100

Activity

Frac

tion

of A

ctiv

ity C

ount

s

Fuzzy Model Neutral Shadow

Fig. 4.12: Log-log plot of the component activity distributions for the fuzzymodel and its neutral shadow.

70


The point at which the two distributions have the same value, �9¢ , is an

activity count that is equally likely to have been chosen from either distri-

bution. This value is used to calculate new activity, �´�� , with ��h to be � ¢and �3� slightly above �D¢ as follows:�9�r58� ¢ �8��79= 7�� | ��,� ×�� xá� ¢ (4.5)

where �,� ×�� is the highest activity value at which either distribution is pos-

itive, and �D¢ is the lowest value at which the two distributions cross. Since

the lowest value at which two distributions cross in Fig. 4.12 is � ¢ 5��3=! |:l7 � and �,� ×"� 5 �3=��#�%$ | :l7 , we set ��hÚ5 � ¢ 5 �9=� | :l7 � and �9� 5ô9= >'&# | :l7 � , respectively.

With these values, we can calculate new activity as in Fig. 4.11 (d). The

new activity values of the fuzzy model is higher than that of the neutral

shadow, which means that new activity signaling the positive adaptability

flows more frequently into the fuzzy model compared to that of the neu-

tral shadow. Therefore, we can conclude from this figure that the fuzzy

model continuously generates new adaptive fuzzy rules much more than

the neutral shadow does.

4.5.2 Adaptive Behaviors

Fig. 4.13 shows 29 salient waves that have pretty much higher activities

than other waves do. The birth and death of a rule can be seen in this

figure. For example, S � occurs before generation 100 and persists for 200

generations. To find out what properties of the salient rules allow them to

71


0 100 200 300 400 500 600 700 800 9000

1000

2000

3000

4000

5000

6000

Generation

Act

ivity

S1

S2

S3

S4

S5

S29

Fig. 4.13: Evolutionary activities of salient waves.

persist longer than other rules, some of them are decoded as in Table 4.2.0 Analysis of Salient Rule S � : In order to identify what behavioral

properties make this rule have high evolutionary activity value, an experi-

ment has been conducted. Fig. 4.14 (a) shows the behaviors of S � , Fig. 4.14

(b) shows speed changes during the simulation, and Fig. 4.14 (c) shows the

activations of S � . As can be seen, S � drives the robot to move forward when

there is no obstacle. As this behavior is important for the robot to move in

open space, S � has been explored at the beginning stage of the evolution.

However, it is interesting that it checks 1 / sensor located in the back of the

robot.0 Analysis of Salient Rule S � : To test the behavioral properties of S � , it

is appended to the previous rule set, which has just one rule, S � . Therefore,

the robot now has two rules, S � and S � as in Table 4.3. Fig. 4.15 (a) shows

the behaviors of S � combined with S � , Fig. 4.15 (b) shows speed changes

72


(a) Behaviors of S ( .

0 5 10 15 20 25 300

5

10

Spe

ed

Left Motor

Right M

otor

0 5 10 15 20 25 300

5

10

Step

(b) Speed changes.

0 5 10 15 20 25 300

0.01

0.02

0.03

0.04

0.05

0.06

Step

Act

ivat

ion

(c) Rule activation.

Fig. 4.14: Behaviors of S � and corresponding data.

73


Table 4.2: Salient rules.

Rule Gene Code MeaningS � 0001010001030411441 ( 1 / 5 F) � ( SRht5 FF) and ( S<�r5 FF)S � 0400010114000110211 ( 1 } 5 VC) and ( 1 / 5 VF)� ( SRht5 S) and ( S<�o5 B)S 0300000410130104031 ( 1 } 5 VF) and ( 1 % 5 C)� ( SRht5 BF) and ( S<�o5 F)S } 0003120304010102401 ( 19�t5 M) � ( SLhm5 FF) and ( S<�o5 BF)S % 0411001112140400311 ( 1��r5 F) and ( 1D i5 F) and ( 1 } 5 M)

and ( 1 % 5 VC)� ( SRht5 F) and ( S<�o5 B)klklk klklk klklkS �*) 1311000003040411021 ( 19ht5 C) and ( 19�r5 F) and ( 1 / 5 F)� ( SRht5 FF) and ( S<�r5 S)

Table 4.3: Test rule set for S � .Rule Meaning

S � ( 1 / 5 F) � ( SLhi5 FF) and ( S��o5 FF)S � ( 1 } 5 VC) and ( 1 / 5 VF) � ( SRht5 S) and ( S<�t5 B)

during the simulation, and Fig. 4.15 (c) shows the two rule’s activations. As

can be seen, the robot becomes to turn left when it meets front obstacles, in

contrast to when it cannot with only S � . The activation of S � is high when

the robot meets frontal walls. The first three left turns are depicted in the

first three peaks of S � in Fig. 4.15 (c), whereas the other peaks of S � shows

repetitive left turns in the bottom center of the simulation environment. S �is also in charge of driving the robot forward. From this analysis, we can

conclude that S � makes the robot to turn left when it encounters a front wall

or obstacle.0 Analysis of Salient Rule S : To test the behavioral properties of S ,74


(a) Behaviors of S (�+ S , .

0 100 200 300 400 500 600−10

−5

0

5

10

0 100 200 300 400 500 600−10

−5

0

5

10

Step

Spe

ed

Left Motor

Right M

otor

(b) Speed.

0 100 200 300 400 500 6000

0.5

1

Act

ivat

ion

S1

S2

0 100 200 300 400 500 6000

0.5

1

Step

S1

S1


Fig. 4.15: Behaviors of S �?� S � and corresponding data.

75


Table 4.4: Test rule set for S .Rule Meaning

S � ( 1 / 5 F) � ( SLhi5 FF) and ( S��o5 FF)S � ( 1 } 5 VC) and ( 1 / 5 VF) � ( SRht5 S) and ( S<�t5 B)S ( 1 } 5 VF) and ( 1 % 5 C) � ( SRht5 BF) and ( S<�r5 F)

it is appended to the previous rule set, which is now increased up to 3 rules,

S � , S � , and S , as in Table 4.4. Fig. 4.16 (a) shows the behaviors of S combined with the other two rules, S � and S � , Fig. 4.16 (b) shows speed

changes during the simulation, and Fig. 4.16 (c) shows the activations of all

the three rules. As can be seen, the robot is now capable of turning right,

which cannot be done with only the other two rules, S � and S � . After four

left turns, which can be identified by the peaks of S � , the activations of S become very high when the robot turns right in the bottom center of the

environment. Careful inspection of the behaviors and the rule S reveals

that the behavior of seemingly turning right is caused by the rule S , which

keeps the robot from departing from right walls.

4.5.3 Evolutionary Pathways to Solutions

In previous sections, we have identified and analyzed the behavioral

properties of some salient rules. The salient rules have important behav-

ioral properties like turn left, keep close to right wall, and move forward.

As these behaviors are important for the robot to navigate safely in the en-

vironment, they are much more adaptive than other behaviors or rules. In

this section, we illustrate that these adaptive behaviors or rules contribute to

the construction of the best individual briefly described in previous section.

76


(a) Behaviors of S ( + S , + S - .

0 100 200 300 400 500−10

−5

0

5

10

0 100 200 300 400 500−10

−5

0

5

10

Step

Spe

ed

Left Motor

Right M

otor

(b) Speed.

0 100 200 300 400 5000

0.5

1

0 100 200 300 400 5000

0.5

1

0 100 200 300 400 5000

0.5

1

Step

Act

ivat

ion

S1

S2

S3


Fig. 4.16: Behaviors of S �?� S �?� S and corresponding data.

77


Table 4.5: Schemata of the salient rules.

Name Gene Code SchemaS � 0001010001030411441 0#0#0#0#0#0#0#11441S � 0400010114000110211 0#0#0#0#140#0#10211S 0300000410130104031 0#0#0#0#10130#0#031S } 0003120304010102401 0#0#120#0#0#0#0#401S % 0411001112140400311 0#110#1112140#0#311klklk klklk klklkS �*) 1311000003040411021 13110#0#0#0#0#11021

Table 4.6: Schemata of the best individual’s rules.

Rule Gene Code SchemaB � 0102041003010411441 0#0#0#100#0#0#11441B � 0400110114000110211 0#0#110#140#0#10211B 0304000410130204031 0#0#0#0#10130#0#031B } 0003120304010103401 0#0#120#0#0#0#0#401B % 0111001102140000311 0#110#110#140#0#311

For this purpose, we utilize schema activities and evolutionary lineage of

the solutions. Note that, from now on, we name the schema of a rule R � as

R ! �l" by surrounding the rule number with braces.0 Rule B and B } : Table 4.5 and 4.6 show the gene codes and their cor-

responding schemata of the salient rules and best individual’s, respectively.

Each schema is obtained by masking off the alleles that do not actually par-

ticipate in the conditional part of the rule. This can be identified by looking

at the “variable toggle flag” in Fig. 4.6.

Comparing the two tables shows direct connections among schemata.

S ! $" is the same with B ! $" and S ! } " is identical to B ! } " , which means that the

rules of S ! $" and S ! } " directly evolved into the rule set of the best individual.

78

4. APPLICATION TO A HARDWARE AGENT0 Other Rules: The construction pathway of B � can be inferred from

the comparison of its schema with other schemata. Comparing S ! �#" with

B ! �#" reveals that they resemble each other very much with differences only

in two positions. Fig. 4.17 (a) shows the schema activity of S ! �#" , and

Fig. 4.17 (b) shows the number of rule instances of the schema. Fig. 4.17

(c) shows the schema activity of B ! �#" , and Fig. 4.17 (d) shows the number

of rule instances of the schema. Here, a schema activity of a schema is a

set of activities of the rules that are instances of the corresponding schema.

From Fig. 4.17 (a) and (b), a lot of activities of S ! �#" can be identified over

all generations, whereas the activity of B ! �#" are relatively very low and sel-

dom in frequency rather than S ! �#" . In addition, the number of instances of

S ! �#" is relatively very higher than that of B ! �#" .Considering the patterns in Fig 4.17 and the schema similarities, it can

be inferred that the rule B � of the best individual is evolved by certain ge-

netic operations, crossover or mutation, on one of the rules of S ! �#" . For

the same reason, B � and B % can also be regarded as descendants of the

instances of S ! �$" and S !#% " , respectively as can be seen in Fig. 4.18 and

Fig. 4.19. Therefore, we can assert that all the rules of the best individual

are from salient rules directly or indirectly. This assertion can be confirmed

by the lineage of the best individual, in Fig. 4.20.

In Fig. 4.20, we can see how the best individual has been evolved

through the evolutionary process. Comparison of rule Á of 21st individual

of generation 1 and rule Á of the best individual reveals how many genetic

mutations have occurred from rule Á of 21st individual of generation 1 to

rule Á of the best individual. Actually, the former rule has experienced

three crossovers that contribute to the formation of the best individual (the

79


0 100 200 300 400 500 600 700 800 90010

0

101

102

103

104

Generation

Sch

ema

Act

ivity

(a) Schema activity of S . (0/ .0 100 200 300 400 500 600 700 800 900

0

10

20

30

40

50

60

Generation

Sch

ema

Inst

ance

s

(b) Number of rule instances of S . (0/ .

0 100 200 300 400 500 600 700 800 90010

0

101

102

103

104

Generation

Sch

ema

Act

ivity

(c) Schema activity of B . (0/ .0 100 200 300 400 500 600 700 800 900

0

10

20

30

40

50

60

Generation

Sch

ema

Inst

ance

s

(d) Number of rule instances of B . (1/ .Fig. 4.17: Analysis of schemata S ! �#" and B ! �#" .

80


100 200 300 400 500 600 700 800 90010

0

101

102

103

104

Generation

Sch

ema

Act

ivity

(a) Schema activity of S . , / .0 100 200 300 400 500 600 700 800 900

0

5

10

15

20

25

30

35

40

45

50

Generation

Sch

ema

Inst

ance

s

(b) Number of rule instances of S . , / .

(c) Schema activity of B . , / .0 100 200 300 400 500 600 700 800 900

0

1

2

3

4

5

6

7

8

9

10

Sch

ema

Inst

ance

s

(d) Number of rule instances of B . , / .Fig. 4.18: Analysis of the schemata S ! �$" and B ! �$" .

81


0 100 200 300 400 500 600 700 800 90010

0

101

102

103

104

Generation

Sch

ema

Act

ivity

(a) Schema activity of S .12 / .0 100 200 300 400 500 600 700 800 900

0

10

20

30

40

50

60

Generation

Sch

ema

Inst

ance

s

(b) Number of rule instances of S .12 / .

0 100 200 300 400 500 600 700 800 90010

0

101

102

103

104

Generation

Sch

ema

Act

ivity

(c) Schema activity of B .12 / .0 100 200 300 400 500 600 700 800 900

0

10

20

30

40

50

60

Generation

Sch

ema

Inst

ance

s

(d) Number of rule instances of B .12 / .Fig. 4.19: Analysis of schemata S !#% " and B !#% " .

82


crossover points are always between the rules). The number of mutations

from the former rule to the latter rule is at least fourteen times because the

former rule and latter rule differ in fourteen alleles. However, in fact, there

might be a great number of mutations in reality from the former rule to the

latter rule.

83


14th individual (780)

0111001112140000311

0003120304010102401

0304000410130204031

0002040003010411441

0400010114000110211

... ...

...

0111001112140410311

...


0003120304020102401

0301100410130104031

...

0111101112140411311

...


...

0003120304020102401

...


0301100410130104031

...

0111001112140000311

...


0003120304010102401

0304000410130204031

...

0002040003010411441

...


0400010114000110211

...

0002030003010411441

...


0400040014040010211

...

0400040014040010211

...


...

0302020112010300001

...

6th individual (6)

...

0114041013001300130

...

33th individual (8)

...

0403120200100210101

...

21th individual (1)

...

0002030003010411441

...


...

1111111100141100131

...

1001010001030411441

27th individual (0)

M

C

M M M

C C

M

M

M

C

M

...

0301100410130104031

...


1

4

4

1

1

1

1

4

4

4

2

2

...

0003120304020102401

...


2

2

2

2

3

3

3

3

3

3

5

5

5

5

5

M Mutation

C Crossover

Best (816th generation)

0111001102140000311

0003120304010103401

0304000410130204031

0102041003010411441

0400110114000110211

... ...

1

2

3

4

5

1 5 ~ Tags for information

Fig. 4.20: The lineage of the best individual.

84


4.6 Emergence Analysis

For the demonstration of observational emergence, we once again state

the parameter settings of emergence.0 First order structures - �2 5 the 4 th fuzzy rule.0 First order interactions Ü,uZ� � 5 the fuzzy inference and defuzzification.0 &+'*) � 5n&+'*) � 5 the behavioral properties of structure.0 Second order structure - � 5 � �T- �2Â. �?&('�) � ��ÜDu�� = the evolved fuzzy

logic controller.

4.6.1 Turning Around

Turning around from a dead end is an important behavior that the robot

should acquire. From the simulation, we find that the three first order struc-

tures - ��. , - �% . , and - �/ . in Table 4.7 are interacting in turning around situa-

tion.

Table 4.7: - ��. , - �% . , and - �/ . .- ��. (=Rule 2) : IF ( 1 } 5 VF)THEN ( SLhi5 FH) and ( S<�o5 F)- �% . (=Rule 5) : IF ( 1 } 5 VC)THEN ( SLhi5 BH) and ( S<�t5 F)- �/ . (=Rule 7) : IF ( 1�ht5 VF) and ( 1 } 5 F) and ( 1 % 5 C)THEN ( SLhi5 BH) and ( S<�t5 F)

Fig. 4.21 shows the &+'*) ’s of the three structures. As you can see,&+'*),�#- ��. shows that the robot stops after some moves from the start po-

sition (see also Fig. 4.22). &+'*),�#- �% . and &+'*),�#- �/ . show that the robot does

85


not move at all. On the other hand, Fig. 4.23 shows &('�) � of the fuzzy con-

troller composed of three structures, - ��. , - �% . , and - �/ . interacting with each

other. During the steps from 1 to 61 in Fig. 4.23, the robot uses all the three

first order structures. Fig. 4.24 shows another view of &+'*) � when the robot

(a) (b) (c)

Fig. 4.21: &+'*) of the three first order structures: (a). &+'*),�#- �� . . (b).&+'*),�#- �%. . (c). &+'*),�#- �/ . .turns around: Fig. 4.24 (a) shows the sensing values of the related sensors

from step 1 to step 100. Fig. 4.24 (b) shows the activation levels of related

rules, 2, 5, and 7, and Fig. 4.24 (c) shows the speed changes of the two

motors during this process.

As can be seen in Fig. 4.24, the interactions of the three first order struc-

tures, - �� . , - �%. , and - �/ . , make the &+'*) � �#- � different from the &('*) � ( - �2 . ) of

first order. This implies thatã Ø�&('*) � �#- � ?� but

ãyæØÓ&('�) � �#- �2 . for all 4#�with

ã 5 “Turn Around”(4.6)

Therefore, we can conclude that the &('�) � �#- � is the emergent behavior by

the definition in equation (3.26).

86


(a) Speed according to the activation ofrule 2.

(b) 354 , 376 , and 3 2 sensor values. (c) Activation of rule 2.

Fig. 4.22: Analysis of &('�) ��. of turning around.

(a) step 1 (b) step 31 (c) step 41 (d) step 61

Fig. 4.23: &('�) � of turning around.

87


Val

ues

Step

d 0 d 4 d 5

100 90 80 70 60 50 40 30 20 10 0 0

200

400

600

800

1000

1200

(a) 3 4 , 3 6 , and 3 2 sensor values.

Act

ivat

ion

Step 0 10 20 30 40 50 60 70 80 90 100

0.0

0.2

0.4

0.6

0.8

1.0 Rule 2 Rule 5 Rule 7

(b) Activation of related rules.

Left Motor Right Motor

100 90 80 70 60 50 40 30 20 10 0 -10

-5

0

5

10

Spee

d

Step

(c) Stepwise speed change of the robot.

Fig. 4.24: Analysis of &+'*) � of turning around.

88


4.6.2 Smooth Cornering

At a corner, the robot should turn the corner to the left or right as safely

and smoothly as possible. From the simulation, we find that the two first

order structures - �� . and - �/ . shown in Table 4.7 are interacting in corner

situations. Fig. 4.25 shows the two structures, &+'*),�#- ��. and &+'*),�#- �/ . . As

(a) (b)

Fig. 4.25: &('*) of the two first order structures of smooth cornering: (a).&+'*),�#- ��. (b). &+'*),�#- �/ . .you can see, &+'*),�#- ��. shows that the robot turns the corner with difficulty,

and also there are some bumps at the corner (see also Fig. 4.26). &('�),�T- �/ . shows that the robot rather moves backward. On the other hand, Fig. 4.27

shows &('�) � of the fuzzy controller composed of two structures - ��. and - �/ .interacting with each other. During the steps from 1 to 90 in Fig. 4.27, the

robot uses all the two first order structures. Fig. 4.28 shows another view

of &('�) � when the robot turns the corner: Fig. 4.28 (a) shows the activation

levels of related rules, 2 and 7, and Fig. 4.28 (b) shows the speed changes

of the two motors during this process.

As can be seen in Fig. 4.28, the interactions of the two first order struc-

tures, - ��. and - �/ . , make the &('�) � �#- � different from the &+'*) � �#- �2À. ) of first

89


(a) Speed according to the activation ofrule 2.

(b) 354 , 376 , and 3 2 sensor values. (c) Activation of rule 2.

Fig. 4.26: Analysis of &('�) ��. of smooth cornering.

(a) step 1 (b) step 28 (c) step 68 (d) step 90

Fig. 4.27: Smooth cornering.

90


0

0.2

0.4

0.6

0.8

0.0 10

1.0

20 30 40 50 60 70 80 90

rule 2 rule 7

Act

ivat

ion

Step

(a) Activation of related rules.

Left Motor Right Motor

Spee

d

Step

0

5

10

-5

-10 0 10 20 30 40 50 60 70 80 90

(b) Stepwise speed change of the robot.

Fig. 4.28: Analysis of &+'*) � of smooth cornering.

91


order. This implies thatã Øz&('�) � �T- � ?� but

ãÛæØÓ&+'*) � �T- �2Â. for all 4Ç�with

ã 5 “Smooth Cornering”(4.7)

Therefore, we can conclude that the &('�) � �T- � is the emergent behavior by

the definition in equation (3.26).

4.7 Summary

In this case study, we have quantified the evolution of a fuzzy logic

controller for a mobile robot and shown that the rules of the best individual

are from their adaptive capabilities. Furthermore, we also have illustrated

the evolutionary construction pathways of the rules of the best individual by

analyzing the schemata of both the rules of the best individual and salient

rules.

The evolutionary activity statistics has shown that the performance of

the best individual is not from other genetic phenomena such as chances

or random genetic drift but from their adaptabilities. Schema analysis has

shown that all the rules of the best individual have been prepared at the ear-

lier stage of evolution and proven adaptive for a long time over generation.

The robot has obtained appropriate rules for several behaviors such as

“smooth cornering” and “turning around” to navigate in the environment.

These rules form the first order structures. We have shown that these first

order structures and their interactions give rise to the emergent behaviors

or observational emergence of second order structures.

92


Therefore, we can conclude that the fuzzy controller has evolved from

adaptive evolution and acquired emergent behaviors. The evolutionary ac-

tivity statistics and schema analysis are proven to be useful for quantifying

adaptability and showing evolutionary construction pathways.

One thing to discuss is the issue of on-line/off-line learning. In on-line

learning, an agent can adapt to changing environments during its life time.

The Michigan approach, one of the GA learning paradigm, or reinforce-

ment learning is frequently used techniques for on-line learning. On-line

learning has advantages in that it enables an agent to learn incrementally to

adapt to the changing environments. However, it has drawbacks in reward-

ing the agent’ actions because, in many cases, the reward can be deferred

or even not given.

In this case study, we have applied off-line learning that enables learn-

ing during the evolutionary process because the experimental environment

does not change during the evolution. Off-line learning does not require

immediate rewards because the performance of an agent can be determined

at the end of test. Therefore, off-line learning can speed up the construc-

tion of the controller and produce better performing controller. However, if

environments change continuously, on-line learning mechanisms should be

considered.

93

CHAPTER 5

APPLICATION TO A

SOFTWARE AGENT

In this chapter, we apply the proposed framework to the construction of

a behavior controller for a software agent. The software agent is a conver-

sation agent that can act as a virtual representative of a web site interacting

with visitors using natural languages.

The agent consists of three main components: dialogue act categoriza-

tion, structured pattern matching, and knowledge construction and repre-

sentation. Dialogue acts (DAs) are identified by automata, which accept

sequences of keywords defined for each of the DAs. We use these DAs to

identify the user’s intention. To make the DA analysis process more ef-

fective, subsumption architecture is used to control the interactions among

the DA analysis modules. Pattern matching is used for the matching the

queries with responses rather than the conventional natural language pro-

cessing techniques, where DA, keywords extracted from DA analysis, and

94

5. APPLICATION TO A SOFTWARE AGENT

Agent Construction

Rule-Based Behavior Controller

Rule-Based Pattern Matcher

Tuning of Internal Parameter by


Genetic Algorithm

Evolution Analysis

Evolutionary Activity

Schema Anaysis



Adaptive Evolution



to Solutions


Behavior Analysis

Conversational Agent

Fig. 5.1: A framework for software agents.

the query are used.

An evolutionary algorithm is applied to the question-answer pattern-

matching module in order to make the matching more flexible as in Fig. 5.1.

The question-answer matching module is implemented based on rules. The

evolved question-answer matcher and the evolution process itself are ana-

lyzed based on the proposed analysis methods.

We apply this agent to the introduction of a web site. The results

show that the conversational agent has the ability to present more adequate,

95


friendly responses.

5.1 Introduction

Since the birth of Internet in 1969, the amount of information has been

increased continuously and the same applies to the each site at which search

engines are used to get the desired information. Most search engines at

present, e.g. YAHOO or Lycos, use keyword-based methods, which are

originate from the beginning of search engine era [59]. As these meth-

ods have difficulties in representation of user’s intention or content, they

produce too many results for users to pinpoint the information they really

want.

Therefore, there is increasing need for the conversational agents that can

identify the user’s intention clearly and thus provide accurate information

and respond quickly with friendly interaction. This case study presents a

conversational agent that is able to have a conversation with a user using

natural languages.

One of the first conversational agents, called Eliza, was born at Mas-

sachusetts Institute of Technology in 1966. Eliza was contrived for the

research on natural language processing. This agent uses simple pattern

matching technique [83]. ALICE (Artificial Linguistic Internet Computer

Entity) is written in a language called AIML (Artificial Intelligence Markup

Language) that is based on XML. It enables other people besides the author

to modify for their personalities. A new idea in ALICE is to tailor the con-

versation for categories of individual, mainly through attempts to determine

the client’s age, gender, geographic location, and occupation. However, it

96


has shortcomings of not being able to respond to users reflecting their in-

tentions because of simple sequential pattern matching that is based on key-

words. Tackling this problem requires much time and effort in constructing

the response database.

In addition to these systems, there are also commercial products. The

Nicole of NativeMinds Corporation acts as a virtual representative for their

site. It provides information on the site to users with natural language with

expressions of his/her state of mind. In addition, there are some products

like SmartBot of Artificial Life Corporation and Verbot of Virtual Person-

alities.

In addition to the application of the proposed framework, the goal of

this case study is threefold: Firstly, we aim to present the DA classifica-

tion mechanism, which identifies the user’s intention and thus facilitate the

matching process. Secondly, the process of structural matching of a query

with a response is presented. Here, by structural matching we mean that

the matching is not just sequential comparisons of components but rather

an evaluation of structural expression, which has the power that is more

expressive in specifying the matching conditions than the sequential one.

Another point we put an emphasis on is that we adopt a pattern-matching

technique rather than the language processing techniques such as parsing or

language generation to overcome the limits of the latter. Lastly, the gram-

mar for the representation of the knowledge base is described.

97


Preprocessing

User Interface Script Generator

Query-Response Matcher

Script Interpreter

Context Manager

Response User Input

Script

Dialogue Act Categorization

Fig. 5.2: Structure of a conversational agent.

5.2 Software Agent: Conversational Agent

The conversational agent we propose: identifies the intention of a query

and returns a response to it in natural language; speaks both Korean and En-

glish; returns different responses to the same query in order to be seemingly

intelligent and diverse.

Fig. 5.2 shows the overall structure of the conversational agent. When

it is preprocessed for the correction of typos and replacement of synonyms,

the query is put into the DA categorization module, which classifies it into

categories of dialogue acts (DAs) and extracts keywords for each DA. This

information, i.e., DA, keywords, preprocessed query, is used to match the

most appropriate response in the script database.

To construct systems like this requires research on three essential tech-

niques. Firstly, the system should be able to classify queries into DAs,

whether it is expressed in Korean or English. This enables the pattern

98


matching to find response close to user’s intention that is roughly expressed

in DAs. Secondly, Pattern matching techniques should be such that users

can get the appropriate response to their query. Lastly, how to represent the

query-response is one of the essential points in constructing the knowledge

for the conversational agents.

5.2.1 Dialogue Acts Classification

If we get to know the intention of a query, then it will be very use-

ful for finding the most appropriate response. A useful approach to this is

the identification of DAs. A DA represents the meaning of an utterance

or a query at the level of illocutionary force [1]. Each query will be as-

signed a unique DA label drawn from a well-defined set (see Table 5.1).

Thus DAs can be thought of as a tag set that classifies queries according

to a combination of pragmatic, semantic, and syntactic criteria [76]. These

DA categories are usually defined to be relevant to a particular application,

although efforts are under way to develop DA labeling systems that are

domain-independent, such as the Discourse Resource Initiative’s DAMSL

architecture [22].

For a conversational agent, it is useful to know whether it is asked a

question or ordered to do something. Queries are classified into two gen-

eral categories, question and statement, which are again sub-categorized

into primary or secondary, each of which consists of several DAs. As a

whole, thirty domain-independent DAs are defined as in Table 5.1. These

DAs enrich the available input for matching a query with a response. An-

other important role of DA information is feedback to lower-level process-

99


Table 5.1: Dialogue acts.

User input Dialogue actPrimary question Can, Who, WhatIf, Method, Location, Reason

Should, Time, Description, Fact, MiscellaneousSecondary question Compare, Confirm, Cost, Direction, DoHave

Example, More, ObtainPrimary statement Message, Act, Is, Have, Want, Fact, MiscellaneousSecondary statement Cause, Feeling, Time, Conditional

ing such that keyword extraction could be constrained by the recognized

DA.

Dialogue Act Categorization module (see Fig. 5.2) classifies queries

into DAs. Only one DA is assigned to the query in case of primary ques-

tion or statement whereas several DAs are assigned to a query in case of

secondary question or statement. Each question or statement has several

predefined DAs as in Table 5.1. Automata that are constructed on keywords

and their sequential information implement the Dialogue Act Categoriza-

tion module.

Fig. 5.3 shows automata that classifies an English query into location

DA. A special Meta character ‘#’ is used to allow the ending of a word

to vary according to subject, tense, etc. For example, the “want#” in the

state transition from S8 to S9 represents “want”, “wants”, “wanted”, and

the same.

The classification process of the thirty DAs is as follows. At first, a

query is tested whether it belongs to one of the DAs of primary question

category, and if it does, once again tested for the secondary question cat-

100


Start

Accept

S1

S2

S3

S4

S5 S6

S7

S8

S9

Accept2 Accept3 Trap

do

tell

what, which

where

me

where

you

know

LOCATION

BE

LOCATION/key2=*

like#, want#, need#

*/key+=*

to

*/key+=* */key+=key2+ *

is, are, was, were, am, be been ,being, do

*/key+=*

LOCATION=state, county, city, street, town, neighborhood, county, nation, location

Fig. 5.3: Automata for location question.

101


s

WHO

ABILITY

MISCELLANE OUS

...

s

s

ACT

MESSAGE

MISCELLANE OUS

...

s

CONFIRMATION

COMPARISION

ACQUISITION ...

CONDITION

TIME

FEELING

s

CAUSE s

Primary question categorization

Secondary question categorization

Primary statement categorization

Secondary statement categorization

A query Categories

Keywords

Fig. 5.4: Dialogue acts classification using subsumption architecture.

egory. A query is classified into only one of the DAs of primary category

whereas several DAs of secondary category can be assigned to it.

Chances are that one query can be classified into several primary ques-

tions or statements if the automata are applied to the classification without

any control or interactions between them. The reason is that there are in-

finite queries while the number of keywords that specifies each DA is lim-

ited. This is in contrast to the formalization that only one DA of primary

category should be classified to as the DA of the query. To solve this prob-

lem, subsumption architecture [17] is adopted to structure and control the

interactions of the automata as in Fig. 5.4.

The subsumption architecture proposed by Brooks can represent control

structures wherein each component suppresses or activates the others. The

suppression is symbolized as ‘S’ in a circle and the activation is without

it on an arrow in Fig. 5.4. “CAN” module, therefore, suppresses “WHO”

102


module, which means that a query classified as “CAN” DA cannot be clas-

sified as “WHO” at the same time. Similarly, primary question category

suppresses primary statement category and activates secondary question

category, and primary statement category activates secondary statement cat-

egory.

5.2.2 Knowledge Representation

In order to response to a query, a script database should be constructed

before the conversation. The script consists of a list of query (or condition)

and response pairs. A part of the script grammar is illustrated in Table 5.2

using BNF notation.

Topic is the primary component that represents a query-response pair

in the grammar. A topic begins with TOPIC keyword followed by name

and one or more conditional statements followed by ENDTOPIC keyword.

A conditional statement is represented as IF (condition) THEN (action). A

condition is a Boolean expression composed of the operand and their oper-

ators such as AND and OR. An operand composed of DAs and comparison

functions is evaluated to be true or not true. The comparison functions are

in Table 5.3. Table 5.4 shows an example script. When a user asks the lo-

cation or direction of something and “lab#”, “softcomputing”, or “soft” &

“computing” appear in the query, one of the items below the “SAYONEOF”

is randomly selected and presented as a response to the user.

103


Table 5.3: Comparison functions.

Name FunctionCONTAIN Checks if the specified string is substring of the target stringMATCH Checks if the specified string is the same of the target stringHEARD The same as CONTAIN except that the target string is a user

query

Table 5.4: A part of script.TOPIC ”location of lab”

IF ((?LOCATIONQUESTION OR ?DIRECTIONSQUESTION)AND HEARD (”lab#”, ”softcomputing”, ”soft”&”computing”)))

THENSAYONEOF

ITEM ”It is located at the 3rd engineering building in yonseiuniversity”

ITEM ”529, the 3rd engineering building, 134, yonsei university,shinchon-dong, seodaemoon-gu, seoul”

ITEM ”The 3rd engineering building in yonsei university”DONE

ENDTOPIC

105


Script Interpreter

IF ((?LOCATIONQUESTION OR ?DIRECTIONSQUESTION)

AND HEARD ( “ lab ” , “ softcomputing ” , “ soft ” & “ computing ” )))

AND

Location Question

Directions Question

OR HEARD

Regular Expression

SAYONEOF ITEM “ It is located at the 3rd engineering

building in yonsei university ” ITEM “ 529, the 3rd engineering building,

134, yonsei university, shinchon - dong, seodaemoon - gu , seoul ”

ITEM “ The 3rd engineering building in yonsei university ”

SAYONEOF

It is.. The.. 529..

Condition Action

T O P I C

Fig. 5.5: Internal representation of script.

5.3 Behavior Controller : Rule-based Pattern

Matcher

Scripts that are composed of according to the grammar in Table 5.2 are

interpreted and loaded into memory by script interpreter when the agent

start running. The conditional part in each topic is transformed into a

Boolean expression and the keywords listed in the conditional part are

transformed into a regular expression as in Fig. 5.5.

The procedure of matching a query with a response is as follows. For

all the topics, the conditional part of a topic is compared with the query,

DAs, and the keyword list extracted during DA classification process. This

returns scores of all the topics as a result. Different types of matching com-

106


Table 5.5: Score assignment.Target ScoreDialogue act 2Keyword Number of charactersAND operator Sum of two operands’ scoresOR operator Max of two operands’ scoresNOT operator 0 : if operand is true, else the score of it

ponents, like DAs, keywords, or Boolean operators, are assigned different

scores as in Table 5.5. All the topics assigned scores, the highest topic is

selected to be presented to a user as a response to his query (see Fig. 5.6).

5.4 Experimental Results

To show the conversational capability of the agent, we put it a role of

introducing web site of a certain research laboratory after constructing the

necessary knowledge base or script. Fig. 5.7 shows a response of the agent

to a query concerning the location of the laboratory. Fig. 5.8 shows a some-

what different response to a location question of the same intention. As

can be seen in Fig. 5.7 and Fig. 5.8, the agent shows intelligent behaviors

by responding a bit differently to the queries of the same intention. What

follows next shows a stream of conversation between a user and the agent.

The agent is able to interact with users naturally presenting the requested

information to them.

User : who are you?

Agent : I am a virtual representative of this site

107


Query

TOPIC 1

TOPIC 2

TOPIC 3

TOPIC n …

DA Categorization

Query - Response Matcher

Response Selection

Response

Condition Action

DAs, Keywords, Query

Scores

Match and get scores

Fig. 5.6: Overall response matching procedure.

Fig. 5.7: Location question 1.

Fig. 5.8: Location question 2.

108


to help you find what you want. So, what do

you want to know?

User : where is your laboratory?

Agent : It is located at the 3rd engineering

building in yonsei university

User : hmm, can I ask your name?

Agent : I am Walwalri.

User : I want to know name of your professor

Agent : My professor’s name is Sung-Bae Cho.

õ

5.5 Summary

In this case study, we have constructed a conversational agent that can

give responses to the queries of users in natural language. The agent can

accept a query in both Korean and English. In addition, the agent is able to

give responses more consistent with the intention of users. The consistency

originates from the identification of user’s intention by the classification of

DAs.

Because the queries are expressed in natural languages, the information

and intention of users are not lost such that the agent can give responses that

are more accurate. Furthermore, the conversational agent has an advantage

of making users feel natural and friendly in finding the required information

because of the interactions using natural language.

We have applied pattern matching (PM) rather than traditional natural

language processing (NLP). The NLP approaches divide user’s queries into

109


tokens and grammatically analyze a sequence of tokens to understand the

meaning of the query. However, these approaches suffer from a great of

grammatical exceptions because the complexity of human languages is ex-

tremely high. In addition, this approach also suffers from the construction

of the corpus sets and high computational costs.

In contrast to NLP, the PM approach is based on empirical knowledge

on natural language. This approach utilizes commonly used language pat-

terns and constructs answer databases from the patterns. Actually, it suf-

fers from constructing the databases. However, the databases can be easily

expanded to adopt new language patterns, which is not easy in NLP ap-

proaches. The PM approach also has another advantage in that it requires

less computational time than the NLP approaches because there is no need

to analyze the queries grammatically.

As further work, automatic construction of answer databases from web

pages is necessary to reduce time and effort in the construction process.

Maintaining contextual information during conversation is also another re-

search topic to improve intelligence and contextual interactions. Finally,

giving the initiative to both sides could make the conversation more natural

than the current implementation of initiative given to only users.

110

CHAPTER 6

CONCLUSION

In this dissertation, we have motivated, formulated, and addressed a soft

computing framework for intelligent agents. Furthermore, as case studies

of the framework, a hardware agent and a software agent have been devel-

oped and analyzed.

As the conventional soft computing frameworks have some shortcom-

ings, especially in both evolutionary and behavioral perspectives, this dis-

sertation proposes a soft computing framework for intelligent agents to

overcome the shortcomings of the conventional soft computing frameworks.

Furthermore, the proposed framework is applied to two case studies, hard-

ware agent and software agent, to show the usefulness of the proposed

framework.

The framework consists of two parts. In the construction part, the com-

bination of rule-based systems and evolutionary algorithms is used to con-

struct intelligent agents. In the analysis part, evolutionary activity statistics,

schema analysis, and observational emergence are applied to the analysis

111

6. CONCLUSION

of evolved agents in both evolutionary and behavioral perspectives.

The framework adopts rule-based systems as a behavior controller for

an intelligent agent for two reasons. First, expert knowledge can be easily

incorporated into the systems and represented in verbal forms. This implies

that it is pretty much easier to analyze the rule-based systems. Second, even

when combined with evolutionary algorithms, the degree of facility in an-

alytic perspective does not change so that the system can be easily dig into

to figure out what, how, when, and why the rules have evolved successfully

or not. These aspects of facility of analysis matches best with the analysis

methods to remedy the drawbacks of the conventional frameworks. To tune

the internal parameters of the rule-based systems, the framework utilizes

genetic algorithms. Genetic algorithms help us to design the rule-based

systems by evolving a near-optimal set of parameters.

In addition to the construction methods of the framework, it has three

analysis methods used to analyze the evolution and the evolved agents.

Evolutionary activity statistics is to measure the capacity to produce good

solutions via evolution. It is an objective, empirical measure of the level of

evolutionary activity in an artificial or natural system. Emergence is used as

a name for creation of new structures and properties. A property of a struc-

ture is observational emergence if there is a deductional or computational

process or theory such that the property cannot be determined by the pro-

cess or theory from the properties or interactions of the components of the

structure. Schema is a similarity template representing a subset of strings

with similarity at certain string positions of a chromosome. Schema analy-

sis is used to identify adaptive behaviors that contribute to the formation of

the solutions.

112

6. CONCLUSION

The framework is applied to two case studies, hardware agent and soft-

ware agent, to show its usefulness. In the case study of hardware agent, we

have quantified the evolution of a fuzzy logic controller for a mobile robot

and shown that the rules of the best individual are from their adaptive capa-

bilities. Furthermore, we also have illustrated the evolutionary construction

pathways of the best individual by analyzing the schemata of both the rules

of the best individual and salient rules.

Evolutionary activity statistics has shown that the performance of the

best individual is not from other genetic phenomena such as chances and

random genetic drift but from their adaptabilities. Schema analysis has

shown that all the rules of the best individual have been prepared at earlier

stage of evolution and proven adaptive for many generations.

The robot has obtained appropriate rules for several behaviors such as

“smooth cornering” and “turning around” to navigate in the environment.

These rules form the first order structures. We have shown that these first

order structures and their interactions give rise to the emergent behaviors

or observational emergence of second order structures.

In the case study of software agent, we have constructed a conversa-

tional agent. The agent is composed of three main components: dialogue

act classification using automata for identifying user’s intention, question-

answer matcher based on pattern matching technique, and knowledge-representation

scheme for the knowledge base for answers.

The agent can give responses to the queries of users in both Korean

and English. Furthermore, the agent can response more consistently with

the intention of users. The consistency originates from the identification of

user’s intention by the classification of DAs.

113

6. CONCLUSION

Because the queries are expressed in natural languages, the information

and intention of users are not lost such that the agent can give accurate re-

sponses. Furthermore, the conversational agent has an advantage of making

users feel natural and friendly in finding the required information because

of the interactions using natural language.

This dissertation has contributed to the followings. First, this disser-

tation has presented a soft computing framework for intelligent agents by

augmenting conventional soft computing frameworks. Compared with the

conventional frameworks, the framework puts an emphasis on the analysis

of the behaviors of the evolved controller and the evolution. Application

of the framework to two real world agents shows that the framework has

shown good results with advantages in analysis than others.

Second, this dissertation has presented a quantitative measure, as a part

of the framework, for adaptive evolution and applied it to two real-world

problems. With this quantitative measure, we can say whether an evolution

has been adaptive more objectively and thus scientifically. Although the

measure is not originated from us, it is our contribution to incorporate it in

the proposed soft computing framework to overcome the shortcomings of

the conventional frameworks.

Third, this dissertation also has presented a method to show the evo-

lutionary pathways to the solutions by combining the evolutionary activity

statistics and schema analysis. Experimental results of case studies show

that the evolved solutions have resulted from the adaptability of each com-

ponent that constitutes the solutions rather than from chances or other evo-

lutionary phenomena. Furthermore, the reason that certain genes are more

adaptive than others is presented by analyzing their behaviors.

114

6. CONCLUSION

Finally, the framework can tell whether a behavior is emergent. We

have formulated the application of observation emergence to a behavior

controller and shown that the evolved controller has obtained emergent be-

haviors during the evolution. We have also shown that the emergent behav-

iors are from the interactions of lower level components.

Consequently, we have confirmed that the proposed framework can be

effectively applied to the construction of controllers for intelligent agents.

Particularly, it has advantages in analyzing the behaviors of an evolved con-

troller and in analyzing the evolution.

115

BIBLIOGRAPHY

[1] Austin, J.L., “How to do things with words,” Clarendon Press, Ox-

ford, 1962.

[2] Aguirre, E. and Gonz 8� lez, A. “Fuzzy behaviors for mobile robot nav-

igation: design, coordination and fusion,” International Journal of

Approximate Reasoning, vol. 25, pp. 255-289, 2000.

[3] Bass, N.A., “Emergence, hierarchies, and hyperstructures,” Artificial

Life 3, pp. 515-537, 1992.

[4] Bayes, T., “An essay towards solving a problem in the doctrine of

chances,” Philosophical Transactions of the Royal Society of London,

53, pp. 370-418, 1963.

[5] Bedau, M.A., “Weak emergence,” Philosophical Perspectives: Mind,

Causation, and World, vol. 11, pp. 375-399, 1997.

[6] Bedau, M.A. and Packard, N.H., “Measurement of evolutionary ac-

tivity, teleology, and life,” In: Artificial Life 2, Addison-Wesley, Red-

wood City, CA, pp. 431-461, 1992.

116

BIBLIOGRAPHY

[7] Bedau, M.A., Snyder, E., and Packard, N., “A classification of long-

term evolutionary dynamics,” Artificial Life 6, pp. 228-237, 1998.

[8] Bedau, M.A., “Quantifying the extent and intensity of adaptive evo-

lution,” Proceedings of the 1999 Genetic and Evolutionary Computa-

tion Conference Workshop Program, pp. 34-37, 1999.

[9] Belgrave, M., “The unified agent architecture,” A White Paper

(http://www.ee.mcgill.ca/elmarc/uua paper.html), 1995.

[10] Berlekamp, E.R., Conway, J.H., and Ray, R.K., “Winning ways for

your mathematical plays,” Academic Press, vol. 2, pp. 817-850, 1982.

[11] Bonarini, A., “Evolutionary learning, reinforcement learning, and

fuzzy rules for knowledge acquisition in agent-based systems,” Pro-

ceedings of the IEEE, vol. 89, no. 9, pp. 1334-1346, 2001.

[12] Bonissone, P.P., “Soft computing: the convergence of emerging rea-

soning technologies,” Soft Computing, vol. 1, no. 1, pp. 6-18, 1997.

[13] Booker, L.B., Goldber, D.E., and Holland, J.H., “Classifier systems

and genetic algorithms,” Artificial Intelligence, vol. 40, pp. 235-282,

1989.

[14] Box, G.E.P. “Evolutionary operation: A method for increasing indus-

trial productivity,” Appl. Statistics, vol. 6, no. 2, pp. 81-101, 1957.

[15] Bremermann, H.J. “Optimization through evolution and recombina-

tion,” Self-Organizing Systems, 1962.

117

BIBLIOGRAPHY

[16] Brenner, W., Zarnekow, R., and Wittig, H., Intelligent software

agents: Foundations and applications, Springer-Verlag, New York,

1998.

[17] Brooks, R.A.: “A Robust layered control system for a mobile robot,”

IEEE Journal of Robotics and Automation, pp. 14-23, 1986.

[18] Carlsson, C. and Walden, P., “Intelligent systems and soft computing,”

Proceedings of the 33rd Annual Hawaii International Conference on

System Sciences, pp. 557-557, 2000.

[19] Chen, C.-C.J. and Miikkulainen, R., “Creating melodies with evolv-

ing recurrent neural networks,” Proceedings. IJCNN ’01, vol. 3, pp.

2241-2246, 2001.

[20] Cho, S.-B. and Ray, T.S., “An evolutionary approach to program

transformation and synthesis,” International Journal of Software En-

gineering and Knowledge Engineering, vol. 5, pp. 179-192, 1995.

[21] Cilliers, P., “What can we learn from a theory of complexity?” Emer-

gence, vol. 2, no. 1, pp. 23-33, 2000.

[22] Core, M.G., Allen, J.F.: “Coding Dialogs with the DAMSL Annota-

tion Scheme,” Working Notes of the AAAI Fall Symposium on Com-

municative Action in Humans and Machines, pp. 28-35, 1997.

[23] Crutchfield, J. P., “Is anything new?” Sciences of Complexity 9;:59 ,

Addision-Wesley, 1994.

118

BIBLIOGRAPHY

[24] Dempster, A.P. “Upper and lower probabilities induced by a mul-

tivalued mapping,” Annals of Mathematical Statistics, vol. 38, pp.

325-339, 1967.

[25] Eccles, J.S., Dickerson, J.A., and Shao, J., “Evolving a virtual ecosys-

tem with genetic algorithms,” Proceedings of the 2000 Congress on

Evolutionary Computation, vol. 1, pp. 753-760, 2000.

[26] Emmeche, C., Koppe, S., and Stjernfelt, F., “Explaining emergence:

Towards an ontology of levels,” Journal for General Philosophy of

Science, vol. 28, pp. 83-119, 1997.

[27] Floreano, D. and Mondada., F., “Automatic creation of an au-

tonomous agent: Genetic evolution of a neural-network driven robot,”

From animals to Animats 3, MIT Press/Bradford Books, 1994.

[28] Fogel, L.J. “Autonomous automata,” Ind. Res., vol. 4, pp. 14-19,

1962.

[29] Franklin, S. and Graesser, A., “Is it an agent or just a program? A

taxonomy for autonomous agents,” Proceedings of the Third Inter-

national Workshop on Agent Theories, Architectures, and Languages,

Springer-Verlag, 1996.

[30] Friedberg, R.M. “A learning machine: Part 1,” IBJ J., vol. 2, no. 1,

pp. 2-13, 1958.

[31] Fukuda, T. and Kubota, N., “An intelligent robotic system based on a

fuzzy approach,” Proceedings of the IEEE, vol. 87, no. 9, pp. 1448-

1470, 1999.

119

BIBLIOGRAPHY

[32] Furuhashi, T., “Fusion of fuzzy/neuro/evolutionary computing for

knowledge acquisition,” Proceedings of the IEEE, vol. 89, no. 9, pp.

1266-1274, 2001.

[33] Gilbert, D., Aparticio, M., Atkinson, B., Brady, S., Ciccarino, J.,

Grosof, B., O’Conner, P., Ostsek, D., Pritko, S., Spagna, R., and Wil-

son, L., IBM Intelligent Agent Strategy, IBM Corporation, 1995.

[34] Glickman, M.R. and Sycara, K., “Evolvability and static vs. dynamic

fitness,” Proceedings of the Evolvability Workshop at the Seventh In-

ternational Conference on the Simulation and Synthesis of Living Sys-

tems, pp. 37-40, 2000.

[35] Goldberg, D.E., Genetic Algorithms in Search, Optimization < Ma-

chine Learning, Addison Wesley, 1989.

[36] Grefenstette, J.J., Genetic Algorithms for Machine Learning, Kluwer

Academic Publishers, Boston, 1995.

[37] Hagras, H., Callaghan, V., and Collry, M., “Outdoor mobile robot

learning and adaptation,” IEEE Robotics & Automation Magazine,

vol. 8, no. 3, pp. 53-69, 2001.

[38] Herrera, F., Lozano, M., and Verdegay, L., “Tuning fuzzy logic con-

trol by genetic algorithms,” Int. Journal Approximate Reasoning,

12(3/4), pp. 299-315, 1995.

[39] Holland, J.H. “Outline for a logical theory of adaptive systems,” J.

Assoc. Comput. Mach., vol 3, pp. 297-314, 1962.

120

BIBLIOGRAPHY

[40] Holland, J.H. and Reitman, J.S., “Cognitive systems based on adap-

tive algorithms,” Pattern-Directed Inference Systems, Academic

Press, 1979.

[41] Holland, J.H., Adaptation in Natural and Artificial Systems, MIT

Press/Bradford Books, 1992.

[42] Holland, J.H., Booker, L.B., Colombetti, M., Dorigo, M., and Gold-

berg, D.E., “What is a learning classier system?” Lecture Notes in

Articial Intelligence LNAI 1813, Springer Verlag, pp. 3-22, 2000.

[43] Hunter, A. and Chiu, K.-S., “Genetic algorithm design of neural net-

work and fuzzy logic controllers,” Soft Computing, vol. 4, no. 3, pp.

186-192, 2000.

[44] Izumi, K., Watanabe, K., and Miyazaki, T., “Fuzzy behavior-based

control for a miniature mobile robot,” Proceedings of Knowledge-

Based Electronic Systems, vol. 3, pp. 483-490, 1998.

[45] Jang, J.S.R., Sun, C.T., and Mizutani, E., Neuro-Fuzzy and Soft Com-

puting, Prentice-Hall, 1997.

[46] K-Team, Khepera Simulator Version 5.02 User Manual, 1999.

[47] Kecman, V., Learning and Soft Computing, MIT Press, 2001.

[48] King, P.J. and Mamdani, E.H., “The application of fuzzy control sys-

tem to industrial process,” Automatica, vol. 13, pp. 235-242, 1977.

[49] Kinzel, J, Klawoon, F., and Kruse, R. “Modifications of genetic algo-

rithms for designing and optimizing fuzzy controllers,” Proceedings

121

BIBLIOGRAPHY

of the First IEEE Conference on Evolutionary Computing, pp. 28-33,

1994.

[50] Mirschner, M. and Gerhart, J., “Evolvability,” Proc. Nat. Acad. Sci.,

vol. 95, pp. 8420-8427, 1998.

[51] Kitano, H. “Empirical studies on the speed of convergence of neural

networks training using genetic algorithms,” Eight National Confer-

ence in Artificial Intelligence, pp. 789-796, 1990.

[52] Kovacs, T., “What should a classifier system learn?” Proceedings of

the 2001 Congress on Evolutionary Computation, vol. 2, pp. 775-782,

2001.

[53] Lee, M.A. and Tagaki, H. “Dynamic control of genetic algorithm

using fuzzy logic techniques,” Proceedings of the fifth International

Conference on Genetic Algorithms, pp. 76-83, 1993.

[54] Lee, S.-I. and Cho, S.-B., “Emergent behaviors of a fuzzy sensor-

motor controller evolved by genetic algorithm,” IEEE Transactions

on System, Man, and Cybernetics: Part B, vol. 31, no. 6, 2001. (To

appear)

[55] Liu, B.-D., Chen, C.-Y., and Tsao, J.-Y., “Design of adaptive fuzzy

logic controller based on linguistic-hedge concepts and genetic algo-

rithms,” IEEE Transactions on Systems, Man and Cybernetics, Part

B, vo. 31, no. 1, pp. 32-53, 2001.

122

BIBLIOGRAPHY

[56] Maley, C.C., “Four steps toward open-ended evolution,” Proceedings

of the Genetic and Evolutionary Computation Conference, pp. 1336-

1343, 1999.

[57] Mamdani, E.H. and Assilian, S., “An experiment in linguistic syn-

thesis with a fuzzy logic controller,” International Journal of Man-

Machine Studies, vol. 7, no. 1, pp. 1-13, 1975.

[58] Mayr, E., Towards a New Philosophy of Biology, Harvard University

Press, 1988.

[59] McBryan, O.A., “GENVL and WWWW: Tools for taming the web,”

Proceedings of the First International Conference on the World Wide

Web, pp. 58-67, 1994.

[60] Melin, P. and Castillo, O., “Intelligent control of complex electro-

chemical systems with a neuro-fuzzy-genetic approach,” IEEE Trans-

actions on Industrial Electronics, vol. 48, no. 5, pp. 951-955, 2001.

[61] Morgan, C.L., Emergent Evolution, Williams and Norgate, 1923.

[62] Nwana, H.S., “Software agents: An overview,” Knowledge Engineer-

ing Review, vol. 11, no. 3, pp. 205-244, 1996.

[63] Pal, S.K. and Wang, P.P., Genetic Algorithms for Pattern Recognition,

CRC Press, 1996.

[64] Pearl, J. “Evidential reasoning under uncertainty,” Exploring Artificial

Intelligence, pp. 381-418, 1988.

123

BIBLIOGRAPHY

[65] Rechenberg, I. “Cybernetic solution path of an experimental prob-

lem,” Royal Aircraft Establishment, Library translation No. 1122,

1965.

[66] Rechtsteiner, A. and Bedau, M.A., “A generic neutral model for mea-

suring excess evolutionary activity of genotypes,” Proceedings of the

Genetic and Evolutionary Computation Conference, vol. 2, pp. 1366-

1373, 1999.

[67] Reynolds, C.W., “Flocks, herds, and schools: A distributed behavioral

model,” Computer Graphics, vol. 21, pp. 25-34, 1987.

[68] Ronald, E.M.A., Sipper, M., and Capcarrere, M.S., “Design, obser-

vation, surprise! A test of emergence,” Artificial Life, vol. 5, pp.

225-239, 1999.

[69] Seng, T.L., Khalid, M.B., Yusof, R., “Tuning of a neuro-fuzzy con-

troller by genetic algorithm,” IEEE Transactions on Systems, Man,

and Cybernetics-Part b: Cybernetics, vol. 29, no. 2, pp. 226-236,

1999.

[70] Shafer, G. A Mathematical Theory of Evidence, Princeton University

Press, 1976.

[71] Shannon, T., “Generic behavior in the Lindgren non-spatial model

of iterated two player games,” Artificial Life 6, MIT Press/Bradford

Books, pp. 316-325, 1998.

[72] Shoharn, Y., “An overview of agent-oriented programming,” Software

Agents, 1997.

124

BIBLIOGRAPHY

[73] Smith, J.M., The Theory of Evolution, New York, 1975.

[74] Smith, S.F., A Learning System Based on Genetic Adaptive Algo-

rithms, PhD thesis, University of Pittsburgh, 1980.

[75] Stewart, I., “The ultimate in anti-particles,” Scientific American, vol.

271, pp. 104-107, 1994.

[76] Stolcke, A., Ries, K., Coccaro, N., Shriberg, E., Bates, R., Jurafsky,

D., Taylor, P., Martin, R., Van Ess-Dykema, C., Meteer, M.: “Di-

alogue act modeling for automatic tagging and recognition of con-

versational speech,” Computational Linguistics, vol. 26, no. 3. pp.

339-373, 2000.

[77] Tan, C.L. and Chia, H.W.K, “Genetic construction of neural logic net-

works,” Proceedings of the International Joint Conference on Neural

Networks, vol. 1, pp. 732-737, 2001.

[78] Taylor, T., “Some representational and ecological aspects of evolv-

ability,” Proceedings of the Evolvability Workshop at the Seventh

International Conference on the Simulation and Synthesis of Living

Systems, pp. 41-44, 2000.

[79] Tettamanzi, A. and Tomassini, M., Soft Computing: Integrating Evo-

lutionary, Neural, and Fuzzy Systems, Springer-Verlag, 2001.

[80] Tsourveloudis, N.C., Valavanis, K.P., and Hebert, T., “Autonomous

vehicle navigation utilizing electrostatic potential fields and fuzzy

logic,” IEEE Transactions on Robotics and Automation, vol. 17, no.

4, pp. 490-497, 2001.

125

BIBLIOGRAPHY

[81] Ulrich, B. and Francisco H., “Ten lectures on genetic fuzzy systems,”

Technical Report of University of Granada, SCCH-TR-0021, 1997.

[82] Wagner, G.P. and Altenberg, L., “Complex adaptations and the evo-

lution of evolvability,” Evolution, vol. 50, pp. 967-976, 1996.

[83] Weizenbaun, J., “ELIZA - a computer program for the study of natural

language communication between man and machine,” Communica-

tions of the ACM, vol. 9, no. 1, pp. 36-45, 1965.

[84] West-Eberhard, M.J., “Adaptation: current uses,” Keywords in Evo-

lutionary Biology, Harvard University Press, pp. 13-18, 1992.

[85] Wilson, S.W., “Mining oblique data with XCS,” Technical Report

2000028, University of Illinois at Urbana-Champaign, 2000.

[86] Winer, G., Periaux, J., Galan, M., and Cuesta, P., Genetic Algo-

rithms in Engineering and Computer Science, John Wiley & Sons,

New York, 1995.

[87] Wong, S.K.M., Butz, C.J., and Wu, D., “On the implication problem

for probabilistic conditional independency,” IEEE Transactions on

Systems, Man and Cybernetics, Part A, vol. 30, no. 6, pp. 785-805,

2000.

[88] Wooldridge, M.J. and Jennings, N.R., “Agent theories, architectures,

and languages: A survey,” Intelligent Agents: ECAI-94 Workshop on

Agent Theories, Architectures, and Languages, pp. 1-39, 1995.

126

BIBLIOGRAPHY

[89] Yager, R.R. and Filev, D.P., “A simple adaptive defuzzification

method,” IEEE Transactions on Fuzzy Systems, vol. 1, no. 1, pp.

69-78, 1993.

[90] Zadeh, L.A., “Fuzzy sets,” Information and Control, vol. 8, pp. 338-

353, 1965.

[91] Zadeh, L.A., “Outline of a new approach to the analysis of com-

plex systems and decision processes,” IEEE Transactions on Systems,

Man, and Cybernetics, vol. 3, no. 1, pp. 28-44, 1973.

[92] Zadeh, L.A., “A theory of approximate reasoning,” Machine Intelli-

gence, pp. 149-194, Halstead Press, 1979.

[93] Zadeh, L.A. “Fuzzy logic and soft computing: issues, contentions and

perspective,” Third International Conference on Fuzzy Logic, Neural

Nets and Soft Computing, pp. 1-2, 1994.

127

�±�%K�³À»ÈÐ

m��æ ÌfCUcl�¦�>çÃ��·�Dø5��¿è«çÃ�É�ÐÚ�b�'�×ß��Òeµ:�¿è«çÃM(#aÿ? �Þ«M(#aUcl�¦�>çÃUc�+�\�£�

��çß�õ�(��ÉÓ'�_�\V��©� ñ��6 x~½Ód��Ér ú§�Érr�çß�õ��6 x��_� �§4��¦

½¹כ�%i�Ü¼ 9ô ¥º��÷&t�·ú§�Ér��6 x��Ð�� ú§�Ér#Q�9¹¡§�¦��>� �%i�

��.\�s��àÔ\�@/ô�Çþj��H_��½[þt�Ér�èáÔàÔ(��ÉÓh�Al�ZO��¦s�6 x �#�

s��Qô�Çë�H]j\�¦K�� HDh�Ðî�rU��¦]jr� �%i��.

�èáÔàÔ(��ÉÓh�Al�ZO��¦&h�6 xô�Ç\�s��àÔ½»¡¤\��H¿º��t�ÅÒ¹כô�Ç~½Ó

ZO��:rs�s�6 x÷&#QM®o��. ��Hy��èáÔàÔ(��ÉÓh�Al�ZO��¦>hZ>�&h�Ü¼�Ð&h�

6 x ��H��¦�\�'�s��.s�~½ÓZO��Ér&h�6 x�)al�ZO�_�?/ÂÒ��pכ��&ñ �l��

#Q§>��Héß�&h�s�e��.��Ér ��H¿º>h ¢��HÕªs��©�_��èáÔàÔ(��ÉÓ

h�Al�ZO�[þt�¦��½+Ë �#�?/ÂÒ��p�'��&ñ_�#Q�9¹¡§�¦¢-a�or�v��H�כ

��X<, :£¤y� ��o·ú��¦o�7£§õ� ��Ér l�ZO�_� ��½+Ës� U�·s�e��>� r��÷&%3�

��.

q�2�¤l��>r~½ÓZO��:rs�$í/BN&h�s�|�� %i�t�ëß�&h�#Q� ��6£§õ�°ú �Ér[j

��t� ë�H]j&h��¦ t�m��¦ e��. 'ÍP:�Ð, ��o�&³�©�\��H &h�6£x$í÷�rëß� ��m�

��Äº��,�9¹כ�,Õªo��¦e��_�_�Ä»��&h�³ðÀÓ°ú �Ér��Ér�&³�©�[þt�¦�í�<Ê �

�¦ e��l� M:ë�H\� ��o��õ� %3�#Q�� K�� &h�6£x$í\�"f %3�#Q�� õ�e��¦

�Ð�©� �t� 3lwô�Ç��. ÑütP:�Ð, l��>r ~½ÓZO��:r�Ér Áº%Ás�, ��]j, #Qb�G>�, Õªo�

�¦�=#Q�"�Ä»��Ha%~�ÉrK��Ð��o ��¦��Ér� Qt�3lw�þt�ÉrÕª]כ��H

t�\�¦�Ðs�t�3lwô�Ç��.s�|9�ë�H[þt�Ér&h�6£xõ��o_�"é¶��¦�Ð��H&h�

\�"f ×�æ¹כ ��. ��t�}��Ü¼�Ð l��>r_� ~½ÓZO��:r�Ér ��o�)a K�\� @/ô�Ç ì�r$3�

128

�¦Ãº'��½+ÉÃº\O��.�9��&h�Ü¼�Ð��o��Héß�í�Hô�Ç ½�\�"f�½Óµ1Ï&h�'��1lx

s��:£¤$íÜ¼�Ð��|cÃºe��H4�¤ú�ô�Ç ½��Ð_��o\�¦�íA�ô�Ç��.Õª

�Q��, l��>r_� ~½ÓZO��:r�Ér �½Óµ1Ï$í\� @/ô�Ç %3��ô�Ç &ñ_�� \O�%3�l� M:ë�H\�

#Q�"�'��1lxs��½Óµ1Ï&h��t�#�ÂÒ\�¦ ½ì�r½+ÉÃº\O�%3��.

s��7Hë�H\�"f��Hl��>r ~½ÓZO��:r_�éß�&h��¦�FG4�¤ �l�0A �#�t�0px+þA\�

s��àÔ\�¦0Aô�Ç�èáÔàÔ(��ÉÓh�A~½ÓZO��:r�¦]jîß�ô�Ç��.s�~½ÓZO��:r�Ér ½»¡¤õ�

ì�r$3�s��H¿ºÂÒì�rÜ¼�Ðs�ÀÒ#Q4Re��. ½»¡¤ÂÒì�r\�"f��H½©gË:l�ìøÍr�

Û¼%7�õ��o·ú��¦o�7£§�¦��½+Ë �#�t�0px+þA\�s��àÔ\�¦ ½»¡¤ô�Ç��.ì�r$3�

ÂÒì�r\�"f��H��o�Ö1lx$í,Û¼v��ì�r$3�,Õªo��¦�'a¹1Ï&h��½Óµ1Ï�¦&h�6 x �#�

��oü<'��1lx&h��'a&h�\�"f��o�)a\�s��àÔ\�¦ì�r$3�ô�Ç��.

��o�Ö1lx$í :�x>��H &h�6£x&h� ��o\�¦ Ãºu�&h�Ü¼�Ð 8£¤&ñ �l� 0Aô�Ç ~½Ó

ZO�Ü¼�Ð"f ��o�� &h�6£x&h��t��t�\�¦ Ì�o�'a&h�s��¦Ãºu�&h�Ü¼�ÐóøÍZ>�

½+ÉÃºe��.Û¼v��ì�r$3��Érþj7áxK��Ð��H��oõ�&ñ_�ì�r$3�\�s�6 x÷&

9�'a¹1Ï&h��½Óµ1Ï�Ér��o�)a]j#Ql�_��½Óµ1Ï&h�'��1lx[þt�¦µ1ßy��HX<&h�6 x�)a

��.

�:r�7Hë�H\�"f��H]jîß��)aáÔYUe��0>ß¼_�Ä»6 x$í�¦·ú��Ðl�0A �#�\�

s��àÔ��½_�¿º��t�ÅÒ¹כâì2£§��èáÔàÔJ?#Q\�s��àÔü< �×¼J?#Q

\�s��àÔ_�¿º��t��YV\�&h�6 xô�Ç��.

'Í��P:��YV��½\�"f��H]jîß�ô�ÇáÔYUe��0>ß¼\�¦ �×¼J?#Q\�s��àÔ

\�¦0Aô�Ç'��1lx]j#Ql�_� ½»¡¤\�&h�6 xô�Ç��.s��YV��½\�"f��6 xô�Ç �

×¼J?#Q \�s��àÔ�Ð��HH�áÔ��¦ ��H {9�7áx_� '��1lxl�ìøÍ s�1lx�Ð4�©�¦

s�6 x �%i��. H�áÔ��_� #�W=� >h ��o�y��t� G'p"f °úכs� '��1lx]j#Ql�_� {9�

§4�Ü¼�Ðs�6 x÷&�¦s��ÐÂÒ'��:r¿º��t�]j#Q°úכs��Ð4�©_�¿º �'�\�¦

½1lx �l�0A �#�s�6 x�)a��.'��1lx]j#Ql��Ð��H½©gË:l�ìøÍr�Û¼%7�_�{9�7áx

��(�t�]j#Ql�\�¦s�6 x ��¦Ä»��·ú��¦o�7£§�¦s�6 x �#�(�t�'��1lx]j

#Ql�_�?/ÂÒ��p�'�\�¦ �&ñô�Ç��.]jîß�ô�ÇáÔYUe��0>ß¼\�¦s�6 x �#��

129

�o�)a]j#Ql�\�@/ô�Ç��ox9�'��1lxì�r$3��¦Ãº'��ô�Ç��.

¿º��P:��YV��½\�"f��H]jîß�ô�ÇáÔYUe��0>ß¼\�¦�èáÔàÔJ?#Q\�s��

àÔ\�¦ 0Aô�Ç '��1lx ]j#Ql�_� ½»¡¤\� &h�6 xô�Ç��. s� ��YV��½\�"f s�6 x÷&

��H�èáÔàÔJ?#Q\�s��àÔ��HR/Û��s�àÔ_��©�@/o��Ð"f��#Q\�¦s�

6 x �#�~½Óë�H��ü<�©� ñ��6 x�¦Ãº'�� H@/�o+þA\�s��àÔs��.s�@/�o

+þA\�s��àÔ��H�o'��ì�rÀÓ, ½�&h�J��B�g�A,Õªo��¦t�d�� ½»¡¤õ�³ð�&³

_�[j��t�ÅÒ¹כ ½$í ��¦�\��_�_��¹�è�Ðs�ÀÒ#Q4Re��Ü¼ 9��6 xכ�

l�0A �#� ��Ð��\�¦s�6 x �#�{9�º��_�v�0>×¼[þt�¦��d��<ÊÜ¼�Ð+��o

'��¦ ì�rÀÓô�Ç��. ¢ô�Ç �Ð�� òÖ�¦&h�� o'��ì�rÀÓ\�¦ 0A �#� �o'��ì�rÀÓ �

Ñýt��s�_��©� ñ��6 x�¦�í[O�½�\�¦s�6 x �#�]j#Qô�Ç��.��6 x��_�|9�_�

\�¦ t�d�� X<s�'� Z�s�Û¼\� e��H ²ú��õ� B�g�A �l� 0AK�"f s� �7Hë�H\�"f

��H��:�x&h��#Q%�o�l�ZO��Ð��HJ��B�g�Al�ZO��¦&h�6 x �%i��.

¿º ��t� ��YV��½\�¦ :�x �#� ]jîß�ô�Ç áÔYUe��0>ß¼�� t�0px+þA \�s��

àÔ\�¦ 0Aô�Ç ]j#Ql�_� ½»¡¤\� òÖ�¦&h�Ü¼�Ð &h�6 x|c Ãº e��6£§�¦ SX�� %i�

��. ¢ô�Ç ��oì�r$3��¦ :�x �#� þj7áxK�� Äº��s�� Ér ��o�&³�©�_� íß�

Óüts� ��m�� y�� ½$í�_¹�èכ &h�6£x$í\�"f q�2�©�)a íß�Óüte��¦ SX��½+É Ãº

e��%3��.��o�)a]j#Ql�\�@/ô�Ç'��1lxì�r$3��¦:�x �#�]j#Ql��oõ�&ñ

\�"f �½Óµ1Ï&h�� '��1lx�¦ S\�1pq �%i�Ü¼ 9, :£¤y� s��Qô�Ç �½Óµ1Ï$ís� �Ð�� ±ú�

�Éréß�>�_� ½$í.��e��¦·ú�Ãºe��%3כ�¹�è[þt_��©� ñ��6 x\�_�ô�Çכ

��:r&h�Ü¼�Ð,]jîß�ô�ÇáÔYUe��0>ß¼��\�s��àÔ_�]j#Ql�\�¦ ½»¡¤ ��H

X< òõ�&h�Ü¼�Ð&h�6 x|cÃºe��6£§�¦·ú�Ãºe��%3�Ü¼ 9,:£¤y��o��õ� ½$í

�)a]j#Ql�_�'��1lxì�r$3�õ��oõ�&ñ\�@/ô�Çì�r$3�\��©�&h��¦t�m��¦e��6£§

�¦·ú�Ãºe��%3��.

Ùþ�d��÷&��H ú�: &h�6£x$í, @/�o+þA \�s��àÔ, ��o �Ö1lx$í :�x>�, ��o ·ú��¦

130

a soft computing framework for intelligent agents: application to

Documents