[IEEE Distributed Processing (IPDPS) - Rome, Italy (2009.05.23-2009.05.29)] 2009 IEEE International Symposium on Parallel & Distributed Processing - Interweaving heterogeneous metaheuristics using harmony search

Download [IEEE Distributed Processing (IPDPS) - Rome, Italy (2009.05.23-2009.05.29)] 2009 IEEE International Symposium on Parallel & Distributed Processing - Interweaving heterogeneous metaheuristics using harmony search

Post on 08-Oct-2016




0 download

Embed Size (px)


  • Interweaving Heterogeneous Metaheuristics Using Harmony Search

    Young Choon Lee and Albert Y. Zomaya Advanced Networks Research Group, School of Information Technologies

    The University of Sydney NSW 2006, Australia



    In this paper, we present a novel parallel-

    metaheuristic framework, which enables a set of heterogeneous metaheuristics to be effectively interwoven and coordinated. The key player of this framework is a harmony-search-based coordinator devised using a recent breed of soft computing paradigm called harmony search that mimics the improvisation process of musicians. For the applicability validation and the performance evaluation, we have implemented a parallel hybrid metaheuristic using the framework for the task scheduling problem on multiprocessor computing systems. Experimental results verify that the proposed framework is a compelling approach to parallelize heterogeneous metaheuristics. 1. Introduction

    Metaheuristics/hyperheuristics are increasingly adopted to deal with many different optimization problems for which no concrete polynomial-time solution algorithms exist. While the primary intention of traditional heuristics is to find a reasonably good solution as fast as possible if necessary at the cost of quality, that of metaheuristics is to effectively search a large solution space in a non-deterministic/stochastic waypossibly spending more timeto find an optimal solution or a near-optimal solution. Specifically, metaheuristics are characterized by their two essential processes: (1) the expansive exploration of search space (diversification), and (2) the guided exploitation of good solutions (intensification). The use of memory is another important characteristic; that is, generally, during the course of searching, best solutions are maintained and constantly updated, and the best solution seen is eventually selected.

    Due to the time complexity of most metaheuristics, in recent years there has been a growing interest in their parallelization [1], [2], [3], [4] (see [5] for further readings on this topic). Although some serious efforts in the hybridization of metaheuristics for parallelization have been made [6], [7], [8], many of these parallel implementation efforts make use of homogeneous algorithms (parallel genetic algorithms (GAs), parallel tabu search (TS) algorithms) [9]. These parallel implementations can be classified as fine-grained or coarse-grained depending on the degree of parallelism; and their communication models are synchronous and/or asynchronous.

    When designing a parallel metaheuristic with heterogeneous algorithms (metaheuristics), the coordinator, or the central communicator, is the key component in that its effective coordination of these algorithms guides them to reach an optimal solution. To this extent, harmony search (HS) [10], a new breed of soft computing algorithm inspired by the improvisation process of musicians, is selected for our framework. The coordination process in our framework is analogous to the process of reaching a best harmony with sounds played by different musicians in the preparation of an improvisation. HS has shown its wide and successful applications in various optimization problems [11], [12], [13].

    This paper presents a novel parallel-metaheuristic framework (PMF), which is composed of a set of heterogeneous worker metaheuristics (WMHs) and a harmony-search-based coordinator (HSC). The interwoven heterogeneous metaheuristics are enabledby the coordinatorto share each others best solutions to help them escape local optima; hence, their solutions are harmonized with each other. For the applicability validation and the performance evaluation, we have implemented a parallel hybrid metaheuristic (HS-PHM) using the framework for the task scheduling problem on multiprocessor computing

  • 1. Set HM to initial (randomly generated) harmonies 2. Create a new harmony h' with harmonies in HM and randomly generated sounds with a probability of pc 3. for each sound si' in h' do 4. Adjust pitch with a probability of with pa 5. if h' is better than the worst harmony hw in HM then

    replace hw by h' 6. Repeat from Step 2 until a termination criterion reaches

    Figure 1. Pseudo-code of HS

    1. Initialize HM with randomly generated harmonies 2. while all harmonies in HM do not have same quality do 3 Wait until a WMH, wmh reports its best solution, s* 4. Replace the worst harmony, hw in HM with s* if s* is better than hw 5. Generate new harmonies using harmonies in HM 6. Adjust pitches of new harmonies 7. Update HM with new harmonies 8. Send a set, H* of best harmonies to wmh 9. end while 10. Send termination signal to all WMHs 11. Select the best harmony in HM

    Figure 2. The workings of HSC

    systems (e.g., computer clusters). We have chosen three representative metaheuristics, a GA, simulated annealing (SA) and artificial immune system (AIS), as worker components. Our evaluation study has been conducted with an extensive set of experiments. Our results clearly show the applicability and practicality of the proposed framework.

    The remainder of this paper is organized as follows. The proposed framework is described in detail in Section 2. Section 3 presents an implementation of our framework for the task scheduling problem. In Section 4, the details of our evaluation study and results are presented. We conclude this paper in Section 5. 2. Parallel-metaheuristic framework

    In this section, we give a brief description of HS, and then present PMF. 2.1. Harmony search

    Musical improvisation is the practice of making and playing music spontaneously. As a short preparation for an improvisation, musicians undergo some trials playing their instruments to find a best harmony. This preparation (HS) consists of a series of processes: harmony generation (randomly or using previous harmonies), pitch adjustment, ensemble inclusion, and dissonance elimination. These processes have similarities to those in other evolutionary algorithmsthe pitch adjustment process can be seen as the mutation operator in GAs. However, HS has several unique features that make it more attractive than other metaheuristics. These features include the generation of a harmony from multiple (previous) harmonies as opposed to two single parent chromosomes in GAs, and the creation of an ensemble between two similar (previous) harmonies.

    A HS algorithm repeatedly constructs harmonies and performs pitch adjustment for each of them until a satisfactory harmony is identified. The construction of harmonies makes use of a harmony memory (HM) in which a number of best harmonies previously found (or random harmonies, initially) are maintained; this

    HM is similar to the memory of musicians. A new harmony is made selecting and combining soundsplayed by different musiciansfrom harmonies in the HM; the selected sounds then have their pitches adjusted for a better harmony. The new harmony replaces the worst harmony in HM, if it is better than the worst harmony. This replacement policy ensures the quality of harmonies in HM.

    A pseudo-code of HS is presented in Figure 1. 2.2. Design description of PMF

    The parallelization of heuristics, specifically metaheuristics, has appeared to be an effective technique in increasing the rate of convergence, and improving the quality of solution. While the first advantage (fast convergence speed) is apparent in parallel metaheuristics, the second advantage may not be easily achievable. This is because the search is often trapped in local optima as in many (serial) metaheuristics. Since the area in the solution space searched by a parallel metaheuristic is likely to be more extensive, this trapping issue can be relieved to a certain degree. The primary design focus of our PMF is the provision of good guidance to parallel metaheuristics for effectively escaping local optima; this eventually leads to find global optima more often.

    PMF is essentially capable of integrating heterogeneous metaheuristics. This integration is enabled by its central componentHSC detailed in Figure 2. The use of heterogeneous metaheuristics implies that various unique features devised in these metaheuristics can be exploited to make the search more efficient. Clearly, the effective coordination between these metaheuristics is the key to achieve good performance. HSC in PMF not only takes charge of this coordination duty, but also accelerates and streamlines the search. Specifically, it initially generates a set of random harmonies and stores them in HM (Step 1). This HM is then updated with best solutions reported from WMHs, and harmonies in HM

  • (a) (b)

    (c) (d)

    (e) (f)

    (g) (h)

    Figure 3. Illustration of a search coordinated by HSC

    are further manipulated using pitch adjustment for improving their quality (Steps 37). Any updates to harmonies in HM are made based on the replacement policy described in Section 2.1. Now, a set of best harmonies in HM is selected and sent to WMHs. HSC terminates if all harmonies in HM have the same quality. Since these metaheuristics have different convergence rates, HSC uses asynchronous communication.

    Figure 3 illustrates how HSC actually coordinates WMHs (GA and AIS in this illustration) and streamlines their searches. Figure 3a shows initial search areas of WMHs. These WMHs report their best solutions once they reach their termination criteria. In our illustration, they both failed to find the global optimum (Figure 3b). HSC allows them to share each others best solutions (Figure 3c). While HSC simply

    sends back best solutions reported from WMHs to themselves without improvements (Figures 3c and 3d), it finds an improved solution and sends it to AIS (Figure 3e); this help from HSC significantly accelerates the overall search (Figures 3f and 3g), and enables WMHs to eventually find the global optimum (Figure 3h). 3. Application of PMF to task scheduling

    The problem of scheduling precedence-constrained tasks on heterogeneous computing systems (e.g., computer clusters) is well known for its NP-hardness; this implies the problem is suitable for evaluation of the applicability and practicality of PMF. In this section, we define the task scheduling problem, and then describe the three WMHs (GA, SA and AIS) developed and adopted as part of HS-PHM. Note that these WMHs are simply our selection from numerous candidate metaheuristics, and their implementations are similar to those presented in our previous work [14]. 3.1. Task scheduling

    Parallel programs, in general, can be represented by a directed acyclic graph (DAG). A DAG, G = (N, E), consists of a set N of n nodes and a set E of e edges (Figure 4). A DAG is also called a task graph or macro-dataflow graph. In general, the nodes represent tasks partitioned from an application; the edges represent precedence constraints. An edge (i, j) E between task ni and task nj also represents inter-task communication. In other words, the output of task ni has to be transmitted to task nj in order for task nj to start its execution. A task with no predecessors is called an entry task, nentry, whereas an exit task, nexit, is one that does not have any successors. Among the predecessors of a task ni, the predecessor which completes the communication at the latest time is called the most influential parent (MIP) of the task denoted as MIP(ni). The longest path of a task graph is the critical path (CP).

    The weight on a task ni denoted as wi represents the computation cost of the task. In addition, the computation cost of the task on a processor pj, is denoted as wi,j and its average computation cost is denoted as

    iw . The weight on an edge, denoted as ci,j represents the

    communication cost between two tasks, ni and nj. However, a communication cost is only required when two tasks are assigned to different processors. In other

  • words, the communication cost when tasks are assigned to the same processor can be ignored, i.e., 0.

    The target system used in this work consists of a set P of p heterogeneous processors/machines that are fully interconnected. The inter-processor communications are assumed to perform with the same speed on all links without contentions. It is also assumed that a message can be transmitted from one processor to another while a task is being executed on the recipient processor which is possible in many systems.

    The earliest start time of, and the earliest finish time of, a task ni on a processor pj is defined as

    =),( ji pnEFT jiji wpnEST ,),( + (2)

    Note that the actual start and finish times of a task

    ni on a processor pj, denoted as AST(ni, pj) and AFT(ni, pj), can be different from its earliest start and finish times, EST(ni, pj) and EFT(ni, pj), if the actual finish time of another task scheduled on the same processor is later than EST(ni, pj).

    Figure 4. A simple task graph The communication to computation ratio (CCR) is a

    measure that indicates whether a task graph is communication intensive, computation intensive or moderate. For a given task graph, it is computed by the average communication cost divided by the average computation cost on a target system.

    The problem addressed in this paper is the scheduling of a set of precedence-constrained (interdependent) tasks, comprising a parallel program, onto a set of tightly coupled heterogeneous processors. The primary goal of this scheduling is to make as many appropriate task-processor matches as possible without violating precedence constraints (data dependencies), so that the overall completion time,

    also called makespan, of a parallel program can be minimized. The makespan in this paper is defined as M=max{AFT(nexit)}the amount of time taken from the time the first task starts running to the time the last task completes its execution. 3.2. Algorithm description

    Since the WMHs of HS-PHM are implemented only as part of our evaluation study (not as the primary work in our study), we use a minimalist approach to their implementations. Clearly, the selection of metaheuristics and their design characteristics substantially affect the performance of parallel metaheuristics developed using PMF.

    In all three implementations, for a given state, one of two mutation/neighbor-selection methods (swap and point) is selected with a probability of 0.5. 3.2.1. AIS

    The immune system is a very complex and yet sophisticated mechanism consisting of a diverse set of entities, principles, and processes. Antibodies and antigens are two key players; and their behaviors and interactions are characterized by different principles and processes, such as negative/positive selection, clonal selection, immune networks, and danger model. An AIS, in general, models one or more entities and processes to deal with real-world problems.

    The AIS (Figure 5) presented in this section is devised using antibodies and clonal selection. The fundamental principle...