Analysis on the Number of XCS Agents inAgent-based Computational Finance

Tomohiro Nakada

Research Center, Institute of Transportation Economics

Email: TomohiroNakada@itej.or.jp

Keiki Takadama

Department of Informatics, University of Electro-Communications

Email: keiki@inf.uec.ac.jp

Abstract—An agent-based simulation developed as a tool toanalyze economic system and social systems since the 1990s.Previous paper reported that the simulation results indicatedthat the number of agents affects the trading prices and theirdistributions. To analyze the effect of the number of agents, thispaper analyzes the relationship between the number of agentsand simulation results using XCS agents for artificial trading. Wereport the market price fluctuation and population size of internalmodel by the number of agents. The revealed the followingremarkable implications: (1) increasing number of XCS agentsdoes not affect the convergence of population size of all agents;and (2) all agents converge towards approximately form 15 %to 20 %of population size by learning classifier system of XCSagents; and (3) increasing number of XCS agents reduce thevariance of the market price.　

Index Terms—Agent-based Computational Finance, XCSagents, Continuous Double-auction Market.

I. INTRODUCTION

Arthur et al. proposed agent-based computational finance in

the stock market [1] [2]. One of the proposed computer models

is a simulator of the economic system creating phenomena

by the interaction of individual autonomic agents, boundedly-

rational agents and learning agents [3]. As one way to analyze

the economic system, agent-based computational finance is

an interdisciplinary study of engineering and economics. In

previous research, many researchers have applied agent-based

computational finance such as the continuous double-auction

market and the financial markets, the foreign exchanges mar-

ket, the carbon dioxide emissions trading markets [4] [5] [6]

[7].Previous work pointed out the different simulation results

by number of agents with a reference [8] [9]. Yamamoto et

al. proposed a massive agent-based simulation on multiple

computers, and evaluated the simulation results by increasing

number of agents [10]. The results indicated high price on

the distribution of the final bid price in auction simulation.

Nakada et al. proposed XCS agents in greenhouse gas emis-

sions trading market. The results indicated big up and down

fluctuation of market price by decreasing number of agents [7].

From these results, We know that the number of agents greatly

influenced the simulation results. To better understand this

agent-based computational finance, it is necessary to evaluate

the simulation results by the number of agents and their

elements.

To compare the effect on simulation results by the number

of agents, this paper proposes a different XCS agent model,

and examines the effect of the number of agents.

The rest of the paper is organized as follows. In section

2, agent modeling as a decision-making model is explained,

Experiment results reported in section 3, and are discussed in

section 4. Our conclusions are presented in section 5.

II. AGENT MODELING

A. Autonomic agent

The autonomic agent is built in a decision-making model

with adaptation and the learning by the development of the

computer science. How do you build a decision-making model

with adaptation and the learning? Arthur defined ”Know”

and ”Learning” of human behavior as ”Recognize data from

environment” and ”Update the data” of model [1]. A learning

classification system and genetic algorithm, the reinforcement

learning are used as a function of the decision-making model

based on this idea.

The previous study proposed a decision-making model using

ZCS which is one of the learning classification systems [5].

Wilson [11] proposed to adapt to complex environments using

accuracy-based fitness. The common point of ZCS and XCS

indicate a combination of state and action using strings of 0, 1

and � (�: only state). The advantage of XCS is a function based

on a measure of the accuracy of the prediction. In complex

phenomena such as market price, it is necessary XCS agent

using the learning and adaptation action using accuracy-based

fitness. Since the function of XCS selects an action based on

the state in environment, this paper assumes as a model of

human behavior.

B. XCS agent modeling

XCS is a learning classifier system using a genetic algorithm

and reinforcement learning showing the action based on an

input state from Environment [11] [12]. This paper explains

brief structure of XCS in Fig. 1. This structure is a system

with the input and output to perform in the next procedure.

1) Detectors perceive an information in environmental, and

convert information into binary number. In this paper, we

define this part in the next section in detail.

8978-1-4673-5921-4/13/$31.00 c©2013 IEEE

Fig. 1. Structure of XCS: that shows a brief decision-making sequence 1-10.

2) Population [P] contains the classifier population in-

cluding state and action of binary number, predictions,

predictions err, fitnesses. This population means the set

of the rule.

3) Match Set [M] take some matched binary number of

detectors from Population [P].

4) If it does not match the classifier population (rule),

covering makes a new rule based on binary number of

detectors.

5) Prediction Array calculates average of prediction based

on the same action of binary number. The maximum

prediction uses for the update of the rule and action

select.

6) Action Set [A] take out a maximum of the prediction as

an action of binary number.

7) Effectors converts action of binary number into a pre-

sented information in the environment. In this paper, we

define this part in the next section in detail.

8) When the action succeed in environment, the reward

feeds back in XCS from environment.

9) Update renew the predictions, predictions err, fitnesses

based on the maximum prediction and the reward.

10) Previous Action [A]−1 renew the classifier population

based on Update.

For more information, other paper explain the detail struc-

ture of XCS [13] [14]. Various methods of XCS are suggested

and improved. But this paper uses basic XCS as decision-

making model.

Fig. 2. Trading Market and Agents in XCS

Fig. 3. State-Action Design

C. Decision-making model

This paper shows a figure of summary of the agent using

XCS in Fig. 2, The individual agents hold one XCS and get

a different rule to get a reward in the trading market. The

agent shows an action (Buying or Selling) based on state

(Information) from trading market as an environment. The

market price indicates a result of interaction of individual

agents. XCS design indicates state and action in several bits.

In artificial market, it is necessary to argue how you design the

input (Detectors) and output (Effectors) of individual agents.

In this paper, the state and action of agent defines 4 bits and 5

bits in Fig. 3. XCS indicate a combination of state and action

using strings of ”0”, ”1” and ”don’t care �” (�: only state).1) Design of State: The detectors of XCS convert a price of

trading market (environment) into binary number. This paper

sets the action division and state division into 200 from 50

such as bid-offer price. This paper defines the relationship

between the price and state 4 bits of binary number in Table.I.In effectors, action of XCS indicates a total of five bits (1

bits is the buying ”0” and selling ”1”, four bits convert market

price) in Table.II.2) Design of Action: Effector changes to trade and price

data from the 5-bit of action selected by the Action Set [A] of

XCS. Design of action described the two versions in the next

sub-section

2013 IEEE Conference on Computational Intelligence for Financial Engineering & Economics (CIFEr) 9

Price State 4 bits Price State 4 bits200 0000 129 – 120 1000

199 – 190 0001 119 – 110 1001189 – 180 0010 109 – 100 1010179 – 170 0011 99 – 90 1011169 – 160 0100 89 – 80 1100159 – 150 0101 79 – 70 1101149 – 140 0110 69 – 60 1110139 – 130 0111 59 – 50 1111

TABLE IDESIGN OF 4 BITS STATE

Action 5 bits Action Price Action 5 bits Action Price00000 Buying 50 10000 Selling 5000001 Buying 60 10001 Selling 6000010 Buying 70 10010 Selling 7000011 Buying 80 10011 Selling 8000100 Buying 90 10100 Selling 9000101 Buying 100 10101 Selling 10000110 Buying 110 10110 Selling 11000111 Buying 120 10111 Selling 12001000 Buying 130 11000 Selling 13001001 Buying 140 11001 Selling 14001010 Buying 150 11010 Selling 15001011 Buying 160 11011 Selling 16001100 Buying 170 11100 Selling 17001101 Buying 180 11101 Selling 18001110 Buying 190 11110 Selling 19001111 Buying 200 11111 Selling 200

TABLE IIDESIGN OF 5 BITS ACTION ”ACTION I ”

3) Design of Reward: The reward gives to the agent who

traded buy and sell. The individual agents compare the trading

price of the artificial market with own price as follows.

Reward ={

TradingP − PriceAagent if supply

PriceAagent − TradingP if demand(1)

In the above equation, TradingP indicates trading price

in market, PriceAagent indicates selected price of buying or

selling based on agent using XCS.

D. Two versions of action design

In order to investigate the simulation results to differences

in the number of agents and action design, this paper sets to

two versions of the design.The first, Action design defines Table II as ”Action I” as

follows. The first two bits indicates buying or selling actions.

The buying action indicates ”00” or ”01” of the first two bits.

The selling action indicates ”10” or ”11” of the first two bits

Since 3 bits of the second half is the price action, the agent

selects the price as follows Table II.The second, Action design defines Table III as ”Action II”

as follows. In effectors, action of XCS indicates a total of five

bits (the buying indicates ”00” of the first two bits, selling

indicates ”01” of the first two bits, and reduction indicates

”10” of the first two bits). ”Reduction” in Table.III is an

element of the model proposed by greenhouse gas emissions

trading market [7]. In order to investigate the influence of the

number of agents and action design, it chose two versions of

these model. The difference between ”Action I” and ”Action

II” is the number of action and selected price.

E. Market design

This paper use the model of double auction market [15]. An

individual agents present selling price and buying price. When

buying price exceeds selling price, a trading price is decided

like Fig. 4. Trading price determined as follows:

Fig. 4. Double auction

TradingP ={

BuyP+SellP2 if BuyP > SellP

0 otherwise(2)

In the above equation, TradingP indicates trading price in

market, BuyP indicates buying price of agents, SellP indicates

selling price of agents.

III. EXPERIMENT

This paper investigates the population size and the market

price by changing the number of agents.

A. Setting

The parameters indicated for agent and XCS as follows. The

number of agents is 2, 10, 30 and 50. Because, market price

showed big fluctuations by decreasing number of agents in

previous work [7]. To investigate the influence of the number

of agents, we analyzed in four cases. The size of state: 4bits,

the size of action: 5bits. N=400, the other parameters used

this paper [13]. We conducted five simulations under these

parameters, and we calculated an average of five times of the

simulation result. In addition, the number of learning trial:

200, the number of dealing trial: 250. Number of these trials

determined based on the results of experiments in Fig. 5.

10 2013 IEEE Conference on Computational Intelligence for Financial Engineering & Economics (CIFEr)

Action 5 bits Action Price Action 5 bits Action Price Action 5 bits Action Cost00000 Buying 50 01000 Selling 50 10000 Reduction 5000001 Buying 70 01001 Selling 70 10001 Reduction 7000010 Buying 90 01010 Selling 90 10010 Reduction 9000011 Buying 110 01011 Selling 110 10011 Reduction 11000100 Buying 130 01100 Selling 130 10100 Reduction 13000101 Buying 150 01101 Selling 150 10101 Reduction 15000110 Buying 170 01110 Selling 170 10110 Reduction 17000111 Buying 190 01111 Selling 190 10111 Reduction 190

TABLE IIIDESIGN OF 5 BITS ACTION ”ACTION II”

Parameter Value

Population size N 400Learning rate β 0.2Discount factor γ 0.71Threshold of GA θGA 50Probability of crossover of the GA X 0.8Probability of mutation in an offspring μ 0.03Value in the second deletion δ 0.1Probability of a � in the condition P� 0.33Prediction in the initial population PI 10Prediction error in the initial population εI 0.0Fitness in the initial population FI 0.01

TABLE IVMAIN PARAMETERS OF XCS

B. Results

Figures 5 and 6 show the average size of population [P] of

XCS of all agents and the market price by the interaction of

individual agents. In Fig. 5, population size in macroclassifiers

(numerosity) decreases by the function in order to eliminate

the same condition and action redundancy as well as reduce

processing time and save storage both Action I and Action II.

Eight results show the stability (Number of trial: 200 - 450)

of the rule set of the state and action using a reinforcement

learning function. The high numerosity of classifier systems

tend to be those with maximal accurate generalizations, so

the most significant knowledge can be determined readily by

sorting the population with numerosity [11].

Fig. 6 shows market price of the period from 200 to 450

in Fig. 5. The configuration of 2, 10, 30 and 50 agents show

the variation between the market prices from 50 to 150. The

market price of Action I shows the variation slightly larger

than the market price of Action II.

To investigate whether the market price is the same fluctua-

tions, we have used statistical analysis to examine the impact

of the number of agents. Fig. 7 and Table.V show the box

-and-whisker plot, the mean and the variance when changing

the number of agents. Table.VI shows Steel-Dwass test in non-

parametric.

IV. DISCUSSION

A. Convergence of population size

As the number of the trials increases, the average population

size of all agents converges towards approximately from 15 %

to 20 % using function of the reinforcement learning [16].

The different number of agents showed differences in the

Fig. 5. Relationship between size trial and size of [P]: the average population[P] of different agents using XCS decreases by trial. The upper figure showsthe simulation results using Action I. The lower figure shows the simulationresults using Action II.

Number of market price market priceAgents Action I Action II

mean sd mean sd2 117.3 15.5 114.1 9.510 112.5 12.3 116.7 12.530 117.5 6.9 117.3 8.450 117.4 5.8 121.3 6.0

TABLE VMEAN AND SD OF MARKET PRICE

convergence process in Fig. 5. Previous paper is discussed the

different convergence point depending on the different sizes

of agent’s action [17]. From these results, if the size of the

action is constant, we can compare the simulation results by

the convergence of the average population size. The average

population size is affected by the action size of the agent

modeling than the number of agents.

B. Market price

Market price fluctuates up and down to around 120 in Fig.

6. Its price shows the middle of the defined price in Table

II depending on supply and demand by the agent’s bidding.

XCS agent acquires by learning and adaptive functions by

a simulation. It is difficult to determine the similarities and

2013 IEEE Conference on Computational Intelligence for Financial Engineering & Economics (CIFEr) 11

Fig. 6. Market price: the market price fluctuated by the interaction of differentagents using XCS. The upper figure shows the simulation results using ActionI. The lower figure shows the simulation results using Action II.

Number of Agents Action I Action IIp-value p-value

2 : 10 0.0045 0.00862 : 30 0.9410 0.00002 : 50 0.8439 0.0000

10 : 30 0.0000 0.982210 : 50 0.0000 0.000030 : 50 0.9995 0.0000

TABLE VISTEEL-DWASS TESTS FOR MARKET PRICE

differences using statistics analysis. This paper analyzes the

relationship between market price and difference in the number

of agents. From Fig. 7 and Table. V, the eight lines of market

price have been small fluctuations. Table. V indicates that

the variance of market price decreases from 15.5 to 6.9 by

increasing number of XCS agents using Action I. Increasing

number of agents is one of the factors that indicate the stability

of market price and equilibrium price. In addition, table. V

indicates that the variance of market price decreases from 12.5

to 6.0 by increasing number of XCS agents using Action II.In addition, since the number of agents (2, 10, 30, 50)

showed outliers in Fig. 7, market price indicates a dynamic

fluctuation by using XCS agent modeling a few agents. In

other words, the forecast of market price will difficult in a

few of homo-agent modeling using agent-based computational

finance.On the other hand, this paper carried out Steel-Dwass test as

a group of each data in Table.VI . The number of agents that

displayed significantly different levels among the groups (p ≤0.05) is three groups in Action I and five groups in Action II.

C. Design of XCS Agent Modeling

In agent-based computational finance, a elements of XCS

agent modeling affect market price sensitivity. First, the num-

ber of XCS agent modeling affects the variance of market

price. Second, the action size of XCS agent modeling affects

the convergence of population size. From these results, the

Fig. 7. box-and-whisker plot of market price: The upper figure shows thesimulation results using Action I. The lower figure shows the simulationresults using Action II.

elements of the XCS agent modeling show an uncertain

phenomenon in different places. For the design of the standard

modeling on XCS agent, we accumulated data from a variety

of conditions of XCS agent modeling and observations of

simulations. We will continue to investigate the behavior of

simulation results and conditions of XCS agent modeling.

We confirmed the strategy(macroclassifiers) depends on the

number of each agent when the population size is stable(Time

= 250). Since each agent has a different strategy, this paper

could not determine the difference in strategy by the number

of agents.

After analyzing the trading volume of the market price, it

was confirmed that the higher the volume by increasing the

number of agents. However, the ratio of the trading volume in

the number of agents showed a value to 0.2 from 0.1.

12 2013 IEEE Conference on Computational Intelligence for Financial Engineering & Economics (CIFEr)

V. CONCLUSION

To investigate the relationship between the number of agents

and simulation results, this paper analyzed simulation results

using the XCS agent modeling of the previous studies [17].

The revealed the following remarkable implications: (1) in-

creasing number of XCS agents, this paper shows that the

variance of the market price decreases. (2) the population size

of all agents converges towards approximately form 15 % to

20 % by increasing number of the trials. Since the number

of agents does not affect the convergence of population size,

it uses the convergence of population size as the evaluation

index of simulation results.

The following issues should be pursued in the near future:

(1) an analysis of population size and market price using

the difference type of XCS agent modeling in agent-based

computational finance; and (2) an application and utilization

of XCS agent modeling.

REFERENCES

[1] W B. Arthur, On Learning and Adaptation in the Economy, In: SantaFe Institute Warking Papers, 92-07-038, 1992.

[2] W B. Arthur, J H. Hooland, B. LeBaron, R. Palmer and P. Taylor, Assetpricing under endogenous expectation in an artificial stock market, In:Santa Fe Institute Warking Papers, 96-12-093, 1996.

[3] L. Tesfatsion, Agent-Based Computational Economics, ISU EconomicsWorking Paper, no.1, pp.1-30, 2003.

[4] S Y. Wong, S. Schulenburg, Portfolio Allocation using XCS Experts inTechnical Analysis, Market Conditions and Options Market, GECCO ’07Proceedings of the 2007 GECCO conference companion on Genetic andevolutionary computation, pp. 2965-2972, 2007.

[5] L. Bull, On using ZCS in a Simulated Continuous Double-AuctionMarket, In W. Banzhaf, J. Daida, A.E. Eiben, M.H. Garzon, V. Honavar,M. Jakiela and R.E. Smith (Eds.) GECCO-99: Proceedings of theGenetic and Evolutionary Computation Conference. Morgan Kaufmann,pp.83-90, 1999.

[6] S H. Chen, M. Kampouridis and E. Tsang, Microstructure Dynamics andAgent-Based Financial Markets, In: The Multi-Agent-Based Simulationworkshop, pp.121-135, 2010

[7] T. Nakada and K. Takadama, Agent-based Modeling Using XCS inGreenhouse Gas Emissions Trading Market, In: SICE Annual Confer-ence 2011, pp.1-5, 2011

[8] B. LeBaron, Agent-Based Computational Finance, In:L.Tesfatsion, K L.Judd (Eds), Handbook of Computational Economics,Volume 2: Agent-Based Computational Economics, Elsevier, pp.1187-1233,2006.

[9] E. Egenter, T. Lux and D. Stauffer, Finite-size effects in Monte Carlosimulations of two stock market models, Physica A, pp.250-256, 1999

[10] G. Yamamoto, H. Tai and H. Mizuta, A Platform for Massive Agent-based Simulation and its Evaluation, In: The Sixth International JointConference on Autonomous Agents and Multi Agent Systems, pp.900-902, 2007

[11] S W. Wilson, Clasifier Fitness Based on Accuracy, Evolutionary Com-putation, vol.3, no.3, pp.149-176, 1995.

[12] F. Kharbat, L. Bull and M. Odeh Revisiting Genetic Selection in the XCSLearning Classifier System, The 2005 IEEE Congress on EvolutionaryComputation, vol.3, pp.2061-2068, 2005.

[13] M V. Butz and S W. Wilson, An algorithmic description of XCS, SoftComputing, vol.6, pp.144-153, 2002.

[14] A. Wada, K. Takadama, K. Shimohara and O. Katai, Analyzing Real-valued Learning Classifier Systems, Transactions of the Japanese Societyfor Artificial Intelligence, vol.20, pp.57-66, 2005. (in Japanese)

[15] D. Friedman and J. Rust, The Double Auction Market: Institutions,Theories, and Evidence, Addison-Wesley, 1993.

[16] R S. Sutton and A G. Barto, Reinforcement Learning: An Introduction,The MIT Press, 1998.

[17] T. Nakada, K. Takadama and S. Watanabe, Decision-making ModelUsing XC in Artificial Market, In: IEEE CEC’2011 Workshop on Agent-Based Economics and Finance, pp.1-5, 2011

2013 IEEE Conference on Computational Intelligence for Financial Engineering & Economics (CIFEr) 13