A “folk” solution to decrease Nash Equilibrium inefficiency for a job scheduling game

1Massimo Orazio Spata, 2Salvatore Rinaudo

1,2 STMicroelectronics, Department: IMS – CAD & Design Services, 1massimo.spata@st.com, 2salvatore.rinaudo@st.com

Abstract

In a distributed system as a Grid, one of the most important targets is to optimize load balancing or thread throughput. As matter of fact in such distributed systems, jobs concur to access shared and distributed resources. For this reason, we modeled a scheduler as a job scheduling game, where multiple jobs concur to use multiple processing cores as players of this game. For each job needs to choose a single core to process it. The payoff of each job is to minimize his job completion time, maximizing the total system throughput. In order to solve this game problem, the integration of Nash Equilibrium and MiniMax solution has been tested in a new algorithm. So the main purpose of this research is to validate the decrease of Nash Equilibrium inefficiency for this proposed distributed model. Experimental tests have focused on the evaluation of different algorithm behavior in order to measure its efficiency.

Keywords: Scheduling problems, Load Balancing, Game Theory, Folk Theorem, Distributed

Systems, Multi-core, Intelligent Complex Systems, Mobile Agents System. 1. Introduction

In the last years, there has been an evolution of computing systems, change from mainframe computing to a distributed client/server approach and continue with network centric computing. The main cause for such trend is mass production of cheap, fast and reliable microprocessors [1]. A network of computer server is becoming a common sight in modern computing environments. However, even in busy environment, a significant proportion of these machines will be idle or underutilized at any time. So a better utilization of the distributed computing environment can be implemented, through help of new scheduling algorithms.

As showed in [2, 3, 12] scheduling algorithms based on Nash Equilibrium help us to achieve the target of maximizing the total throughput in such distributed systems. But, in such situation, like for example the Prisoners’ Dilemma problem, the Nash Equilibrium solution, is not the Pareto optimal solution [13, 15].

In order to solve this problem a new algorithm based on Game Theory has been implemented, merging the MiniMax algorithm with the Nash Equilibrium solutions.

As suggested in [13] for iterated games there is an interesting result in game theory - curiously known as the folk theorem - which shows how the infinitely repeated Prisoners’ Dilemma, under rationality hypothesis, making it possible to obtain an efficient payoff balance for both players. It allows reducing damn of Nash Equilibrium solution inefficiency. Starting from this idea, we try to merge MiniMax solution to Nash Equilibrium solution.

In the section that follows, some concepts of Game Theory with mobile agent are briefly reviewed. Section 3 and Section 4 present the proposed load balancing algorithm and Mobile Agents System designed. Finally, section 5 summarizes the main contributions to this paper and comments on future directions for this work.

2. Related Works

Grid is a heterogeneous distributed system, in terms of hardware and software (such as Operating Systems and applications).

For this reason, often, in a Cluster Grid, users must choose the resources type suitable for the specific job application and for the specific type of operating system. Moreover, users should estimate

A “folk” solution to decrease Nash Equilibrium inefficiency for a job scheduling game Massimo Orazio Spata, Salvatore Rinaudo

Journal of Convergence Information Technology(JCIT) Volume7, Number4, March 2012 doi:10.4156/jcit.vol7.issue4.16

134

the right amount of hardware resources to be reserved for that specific application (such as RAM and CPU time), and finally user should estimate the job deadline [4, 5, 6, 7].

Hence, this Agent system was projected to realize a transparent infrastructure system, in order to automate the process of submission and to maximize the total system throughput. This Agents system has been tested on a Cluster Grid used to simulate microelectronic devices through EDA tools [2, 3, 12]; JADE (a Java Agent Development Framework) has been used through a middleware that complies with the FIPA specifications and through a set of graphical tools that supports the debugging and deployment phases [9, 10]. The main entities pertaining to the projected system architecture are:

· jobInfo: Mysql Database with job information. · The Interpreter Agent · The MiniMax Agent · The Nash Agent · The MOS algorithm. · The Scheduler Agent

In the next sections, the theory on which is based the Agent modeling of the above entities will be

explained.

3. Game Theory: Strategic Game form and Nash Equilibrium definitions

In [13] a strategic game form G, with two players is: (X,Y,E,h)

where:

· X,Y,E are sets · X represents available choices for player I; idem for Y respect to player II. · E represents a set of possible strategies for the game · h : X×Y → E. So h is an output function providing results achieved on players’ choices,

and if player I chooses x and player II chooses y, get h(x,y) results. In a game G, you need to know the players' preferences for the different elements of E. A fast and simple mode to describe these preferences is to use "utility functions" u(x) [11, 12, 13,

14]. For example, for player I assume it is given a function u defined in E and with values in ℝ, interpreting u(e')≥ u(e'') as an expression of: player I prefers outcome e' to outcome e'' [11, 12, 13, 14].

So, having a game form (X,Y,E,h) and two utility functions (for both players) (u1,u2), this is the expression form of the game:

(X,Y,E,h,u1,u2)

Through composition operator ∘, let be f1=u1 ∘ h and f2=u2 ∘ h, we will obtain (X,Y,f1,f2) or a strategic form game for two players, as defined in [4].

Where: · X,Y are sets · f1,f2:X×Y→ℝ · A Nash Equilibrium for G=(X,Y,f1,f2) is (x*,y*) Î X×Y:

o f1(x*,y*)≥ f1(x,y*) " x ÎX

o f2(x*,y*)≥ f2(x*,y) " y ÎY [13]

4. The Multi Agent System: MAS

A “folk” solution to decrease Nash Equilibrium inefficiency for a job scheduling game Massimo Orazio Spata, Salvatore Rinaudo

135

Multi Agent System (MAS) is a branch of applied Distributed Artificial Intelligence. The main problem concerning MAS at design time is to predict all the possible scenarios which would be encountered by MAS during its execution. So they must be dynamic and have the capability to learn from environments, adapting to these different scenarios [11].

Functionality and Quality Goal model Before presenting the Agent Sequence Diagram model, let us introduce the goal model that will be

considered throughout the paper. Functionality Goal: · To minimize the total jobs completion time, maximizing the throughput [2, 3, 12] · To send and receive information regarding submitted jobs · To calculate the input netlist complexity for the EDA simulation [2] · To calculate: initial load, the job service rate λ, and job inter-arrival rate μ [2] · To create the payoff matrix with the above parameters as arrays of strategies [2, 3] · To find the Nash Equilibrium [3, 13, 14, 15] · To find the MiniMax solution · To compare Nash and MiniMax solutions · To schedule jobs on the selected WN

Quality Goal: · To send computed values to a Scheduler Agent

Modeling Agents with UML Sequence Diagram Figure 1 shows the Agent architecture model obtained through the UML Sequence Diagram:

Figure 1. Class Agents UML sequence Diagram

Nash Agent It is an Agent instantiated for every submitted job and analyzes all the possible Nash Equilibrium

solutions. His behavior is defined by NashBehaviour Class. The target of Nash Agent is analyzing an input file containing payoff matrix for the submitted jobs.

The Nash Agent post condition is to invoke start method of FindNashEquilibrium Class in order to find Nash Equilibrium solution for the Prisoners’ Dilemma problem.

A “folk” solution to decrease Nash Equilibrium inefficiency for a job scheduling game Massimo Orazio Spata, Salvatore Rinaudo

136

MiniMax Agent It is an Agent instantiated for every submitted job. The behavior of this Agent is defined by class

MiniMaxBehavior. MiniMax Agent target is to return the MiniMax solution for the matrix M. Interpreter Agent The target of this Agent is to analyze the input netlist file for every simulation job. The Agent

behavior is defined by: InterpreterBehaviour Class. This Agent gives a weight to the job input complexity.

Scheduler Agent The target of this Agent is to schedule jobs on the most appropriate WN, following the

matchmaking of MOS solution values. The Agent behavior is defined by: SchedulerBehaviour Class. 5. Prisoner’s Dilemma

The Prisoner’s Dilemma In the Game Theory, the Prisoner’s Dilemma is a type of non-cooperative game where the only

concern of each individual player ("the prisoner") is to maximize his own payoff, without any concern for the other player's payoff [11].

In this type of game the preference relation effect >i is that players have an obvious strategy to apply, and this is consequent to the presence of dominant strategies. The other effect of payoff is that the result of strategic interaction is non-efficient for this game [12, 13, 14].

Model of agent payoff Apply this model to a specific example and suppose having m players’ equivalent to m jobs that

must be scheduled on t Worker Node WN: · Possible moves are equivalent to all possible job allocations j1, j2 ,…,jm on

WN1 ,WN2,…,WNt · Total available moves number for every agent is given by tm · Payoff in our model is equivalent to minimizing job completion time, optimizing load [2] · Game Matrix M={M1,…,Mp} with i = 1,…,p · A preference relation >i to maximize throughput: d>c>b>a where a,b,c,d are parameters

representing the 1-minute exponentially average CPU run queue length [16]. Briefly, players correspond to jobs. The strategy set of each player is the set of initial load for every

machine. Given a strategy for each player, the total load on each machine is given by the sum of processing times of the jobs that choose that machine.

Usually each player seeks to minimize the total job completion time on its chosen machine. So, standard objective function is minimizing the job completion time on one loaded WN and this objective is called makespan minimization.

For example: given game with 2 Worker Node WN1 and WN2 and 2 jobs j1 and j2. The rows represent the strategies job j1 can choose and the columns represent the strategies job j2 can choose. The following table is a simple Game Matrix M for the Prisoner’s Dilemma problem for 2 agents (jobs) and 2 Worker Node (WN1 and WN2):

M: j2 j1

WN1 WN2 WN1 (c , c) (a , d) WN2 (d, a) (b, b)

Where, for example, in the case of the Prisoner’s Dilemma problem d>c>b>a.

A “folk” solution to decrease Nash Equilibrium inefficiency for a job scheduling game Massimo Orazio Spata, Salvatore Rinaudo

137

On the above Game Matrix, Nash Equilibrium will be calculated as follows: fixed payoff for agents’ j1 then the second agent j2 choosing between the second component of the two strategies on row 1 (c, d) and on the row 2 (a,b), then fixed payoff for agents’ j2 then the first agent j1 choosing between the first component of the two strategies on column 1 (c,d) and column 2 (d, b). The Agents j1 and j2 follow the relation preference to verify if a better strategy exists [11].

Nash Equilibrium Algorithm

· For every Matrix M, o For every payoff array o Compare, the second component of each array with all the other components per

line, o If it is not minimum value then it is not a Nash Equilibrium, o Else if it is the minimum, then take the first component of that array and compare

to all other per column, o If it is the minimum too, then return this Nash Equilibrium solution for the

Prisoner's Dilemma problem

In other words, every Nash Agent will do conjectures about other agents’ strategies and make the best choice (with the highest profit value) for him, ensure that every other agent does not have other strategies with the highest profit, moving on all matrixes and following Nash Equilibrium solutions for Prisoner’s Dilemma problem [13, 15].

MiniMax Algorithm In Game Theory the MiniMax [17], is a strategy to minimize the maximum possible loss, or

alternatively, to maximize the minimum gain (MaxiMin). In zero-sum games, the MiniMax solution is the same as the Nash Equilibrium.

In the case of study, the Prisoners’ Dilemma Game is a non-zero-sum game, so MiniMax Solution, is not necessarily equivalent to Nash Equilibrium Solution.

Follow the implemented MiniMax algorithm: MiniMax Algorithm · For every Matrix M · For every line of the matrix get maximum values of any first component

· Between every maximum values get the minimum value and return the line number for that value

· For every column of the matrix get maximum values of any second component · Between every maximum values get the minimum value and return the column

number for that value · The number of line and column define the payoff array which is the MiniMax solution for that

matrix M of the Prisoner’s Dilemma problem.

A “folk” solution to decrease Nash Equilibrium inefficiency for a job scheduling game Massimo Orazio Spata, Salvatore Rinaudo

138

MOS Algorithm · For every Matrix M

o Calculate every Nash Equilibrium and select the solutions through Nash Agent § If Nash Equilibrium solution not exists then return MiniMax solution for the

initial game matrix o Calculate every MiniMax and select the solutions through MiniMax Agent o Play a new Game between all the selected Nash and MiniMax Agents. o For every Nash Agent compare his solution with a MiniMax Agent solution

§ If the Nash Agent array parameters correspond to the MiniMax Agent array parameters and there are not any other solutions then return this solution

§ Else for every Nash solution, and for every MiniMax solution iterate the Prisoner’s Dilemma problem, building a new 2x2 matrix with Nash solution in the first line and MiniMax solution in the second line (both solutions provided by respective Agents)

· Apply Nash Equilibrium solution to the new Matrix and return this solution.

o Refine obtained solution, playing a new check Game between the returned solution and MaxiMin solution.

7. Experimental results

The job scheduling game simulation is executed with different input matrix parameters, gets from jobs monitoring statistical data. Tests demonstrate the strength of the implemented system. As an initial setup for simulator we choose following settings:

· Number of Worker Nodes: 5 · Number of jobs: 500 · Number of different input: 5

Data for play matrix experimental tests has been collected on 5 different input, with Eldo simulator

[8], allowing the creation of a game with 250 matrixes of different payoffs. The Pareto optimal solution percentage for the experimental job simulations is reported in Fig. 2:

Figure 2. Pareto Optimal solution percentage for the different applied algorithms

A “folk” solution to decrease Nash Equilibrium inefficiency for a job scheduling game Massimo Orazio Spata, Salvatore Rinaudo

139

8. Conclusions

In this paper, we describe our proposed efficient load balancing algorithm and Pareto optimal resource utilization using Mobile Agent. Our proposed algorithm use Mobile Agent MOS algorithm in order to optimize distributes work load, and trying to decrease Nash Equilibrium and MiniMax inefficiency.

Experimental tests demonstrate that implemented system is able to calculate dynamically Pareto Optimal load conditions increasing the total jobs throughput, and with variable load and job/cpu ratio. In figure 2 the optimal percentage for MOS algorithm shows an average improvement in comparison to Nash and MiniMax algorithm.

So, the main target achieved is decreasing significantly MiniMax and Nash Equilibrium inefficiency. According to preliminary results, although our system gives a good result to some extent, there are some points to be modified to reach our target. Therefore we will describe our results in detail in our forthcoming work.

In the near future, this research may be applied to multi-cores systems or to the Smart Grids systems [18, 19, 20]. So, for example using this implemented idea, we are able to predict the behavior of a general distributed system as a Smart Grid. Each agent behaves rationally and only reacts to a price signal, and builds an intelligent Agent-based strategies device system. The described system can learn the best energy storage profile given the market prices that keep changing as a result of consumers using the energy storage.

Acknowledgment Authors wish to thanks their company STMicroelectronics.

9. References [1] Cho Cho Myint; Khin Mar Lar Tun; "A Framework of Using Mobile Agent to Achieve Efficient

Load Balancing in Cluster," Information and Telecommunication Technologies, 2005. APSITT 2005 Proceedings. 6th Asia-Pacific Symposium on , vol., no., pp.66-70, 10-10 Nov. 2005 doi: 10.1109/APSITT.2005.203632

[2] Massimo Orazio Spata, Giuseppe Pappalardo, Salvatore Rinaudo, Tonio Biondi: Agent-Based Negotiation Techniques for a Grid: The Prophet Agents. e-Science 2006: 149.

[3] Massimo Orazio Spata: A Nash-Equilibrium Based Algorithm for Scheduling Jobs on a Grid Cluster. WETICE 2007: 251-252.

[4] I. Foster and C. Kesselman, editors. The Grid (2nd Edition): Blueprint for a New Computing Infrastructure. Morgan Kaufmann, 2004.

[5] D. Minoli. A Networking Approach to Grid Computing. Wiley InterScience, 2005. [6] I. Foster, C. Kesselman, and S. Tuecke. The Anatomy of the Grid: Enabling Scalable Virtual

Organization. International Journal of High Performance Computing Applications, 15(3):200–222, 2001.

[7] Globus Alliance. http://www.globus.org. Globus Home Page, 2004. [8] WWW. Mentor Graphics Corp. Eldo simulator

http://www.mentor.com/products/ic_nanometer_design/custom_design_simulation/eldo/index.cfm.

[9] WWW. JADE a Java Agent Development Framework http://jade.tilab.com/ [10] Foundation for Intelligent Physical Agents, FIPA Agent Management Specification & Support for

Mobility Specification, Geneva, Switzerland, October 2000. Available at: http://www.fipa.org [11] Amjad Mehmood, Abdul Ghafoor, H. Farooq Ahmed, Zeeshan Iqbal, "Adaptive Transport

Protocols in Multi Agent System," itng, pp.720-725, Fifth International Conference on Information Technology: New Generations (itng 2008), 2008

[12] Massimo.Orazio Spata, Salvatore Rinaudo, “A scheduling Algorithm based on Potential Game for a Cluster Grid” JCIT Journal of Convergence Information Technology, (2009)

A “folk” solution to decrease Nash Equilibrium inefficiency for a job scheduling game Massimo Orazio Spata, Salvatore Rinaudo

140

[13] F. Patrone (Author) “Decisori razionali interagenti” Una introduzione alla Teoria dei Giochi - Edizioni PLUS, Pisa, 2006 ISBN: 88-8492-350-6

[14] Martin J. Osborne (Author), Ariel Rubinstein (Author) “A Course in Game Theory” Publisher: The MIT Press (July 12, 1994) – Par. “Strategic Games” Pag. 9, Pag. 11, Pag. 14

[15] John Nash, "Equilibrium points in n-person games", 48-49, Proceedings of the National Academy of Sciences, 36, 1950

[16] Platform LSF Administration guide. Platform LSF web page: http://www.platform.com/Products/platform-lsf

[17] Von Neumann, J: Zur Theorie der Gesellschaftsspiele Math. Annalen. 100 (1928) 295-320 [18] Intelligent multi-player smart grid management considering distributed energy resources and

demand response Vale, Z.A.; Morais, H.; Khodr, H.; Power and Energy Society General Meeting, 2010 IEEE Digital Object Identifier: 10.1109/PES.2010.5590170 Publication Year: 2010 , Page(s): 1 - 7

[19] Optimal and autonomous incentive-based energy consumption scheduling algorithm for smart grid Mohsenian-Rad, A.-H.; Wong, V.W.S.; Jatskevich, J.; Schober, R.; Innovative Smart Grid Technologies (ISGT), 2010 Digital Object Identifier: 10.1109/ISGT.2010.5434752 Publication Year: 2010 , Page(s): 1 - 6

[20] Distributed Demand Management in Smart Grid with a Congestion Game Ibars, C.; Navarro, M.; Giupponi, L.; Smart Grid Communications (SmartGridComm), 2010 First IEEE International Conference on Digital Object Identifier: 10.1109/SMARTGRID.2010.5622091 Publication Year: 2010 , Page(s): 495 - 500

PhD. Massimo Orazio Spata graduated in 1998 in Computer Science, received his PhD degree in Computer Science from Dipartimento di Matematica ed Informatica in 2008. He joined STMicroelectronics Catania in 1999 as CAD Engineer. Since January 2003 Massimo has joined the IMS CAD and Design Services Department. In November 2009 Massimo joined Technical Staff Engineer Staff Group, developing R&D know how and skills in distributed system algorithm. Author of publications and papers for journal and international conferences concerning the above fields. Ing. Salvatore Rinaudo Director of CAD & Design Services Department of Industrial and Multisegment Sector of STMicroelectronics. He received the Laurea degree in nuclear engineering from the University of Palermo, Palermo, Italy, in 1989. He was involved in numerical simulation of semiconductor devices with emphasis on optimization and parameter extraction techniques for process and device design. Since 1991, he has been with STMicroelectronics, Catania, Italy. Since 2000, he has been responsible of Design Methods Developments covering also Process, Device and Physical Modelling across technologies, designs and Data management, Design Services and Knowledge Management.

A “folk” solution to decrease Nash Equilibrium inefficiency for a job scheduling game Massimo Orazio Spata, Salvatore Rinaudo

141