distributing workﬂows over a ubiquitous p2p...

Scientific Programming 15 (2007) 269–281 269IOS Press

Distributing workflows over a ubiquitous P2Pnetwork

Eddie Al-Shakarchia,∗, Pasquale Cozzab, Andrew Harrisona, Carlo Mastroiannic, Matthew Shieldsa,Domenico Taliab and Ian TaylordaSchool of Computer Science, Cardiff University, UKbDEIS University of Calabria, Rende (CS), ItalycCNR-ICAR, Rende (CS), ItalydCenter for Computation and Technology, Louisiana State University, USA

Abstract. This paper discusses issues in the distribution of bundled workflows across ubiquitous peer-to-peer networks for theapplication of music information retrieval. The underlying motivation for this work is provided by the DART project, whichaims to develop a novel music recommendation system by gathering statistical data using collaborative filtering techniques andthe analysis of the audio itsel, in order to create a reliable and comprehensive database of the music that people own and whichthey listen to. To achieve this, the DART scientists creating the algorithms need the ability to distribute the Triana workflowsthey create, representing the analysis to be performed, across the network on a regular basis (perhaps even daily) in order toupdate the network as a whole with new workflows to be executed for the analysis. DART uses a similar approach to BOINC butdiffers in that the workers receive input data in the form of a bundled Triana workflow, which is executed in order to process anyMP3 files that they own on their machine. Once analysed, the results are returned to DART’s distributed database that collectsand aggregates the resulting information. DART employs the use of package repositories to decentralise the distribution of suchworkflow bundles and this approach is validated in this paper through simulations that show that suitable scalability is maintainedthrough the system as the number of participants increases. The results clearly illustrate the effectiveness of the approach.

1. Introduction

This paper describes a novel Music Recommenda-tion System (MRS) that employs the use of the underly-ing Distributed Audio Retrieval using Triana1 (DART)peer-to-peer (P2P) subsystem in order to provide afan-out mechanism for distributing workflows acrossthe network and for retrieval and aggregation of re-sults. Mrs Dart utilises a combination of distribut-ed systems technologies ranging from Grid comput-ing, peer-to-peer, Web Services and workflow. TheUK Science and Technology Facility Council PIPSS2

∗Corresponding author: Eddie Al-Shakarchi, Cardiff School ofComputer Science, Cardiff University, Queen’s Buildings, 5 TheParade, Roath, Cardiff CF24 3AA, UK. Tel.: +44 (0)29 2087 4812;Fax: +44 (0)29 2087 4598; E-mail: [email protected].

1see MRS DART website www.mrsdart.com.2see http://www.pparc.ac.uk.

(PPARC Industrial Programme Support Scheme) fund-ed project at its core is based on the volunteer comput-ing paradigm that typically employs the use of homecomputers for the analysis of data, e.g. radio-telescopedata for SETI@home,3 gravitational wave data forEinstein@home,4 and so on. In the DART scenariohowever, the home users, or workers as they are re-ferred to, perform analysis of their own music collec-tion by executing Triana workflows that encompass theanalysis to be performed at that time.

It is crucial to the project that the DART environmentis capable of self-updating as it is intended not only toprovide music recommendations for end users but alsoto establish itself as a research platform that music in-formation retrieval (MIR) scientists can use in order to

3see SETI@Home website http://setiathome.berkeley.edu/.4see Einstein@Home website http://einstein.phys.uwm.edu/.

ISSN 1058-9244/07/$17.00 2007 – IOS Press and the authors. All rights reserved

270 E. Al-Shakarchi et al. / Distributing workflows over a ubiquitous P2P network

test out new ideas in the MIR field. To this end, scien-tists will want to design new methods for audio analysis,both for statistical correlation based on tags, and moreinterestingly for the analysis of the actual audio itselfto extract things like tempo, pitch, mood and so forth.We believe these high-fi parameters will allow DARTto stand out from the existing recommendation systemsthat generally are based on grouping what people havebought, and do not take into account the actual audionor do they take into account what people actually haveand what they have listened to. Further, we intend tobuild a DART community, rather similar to those host-ed on Web sites like mySpace and Facebook, to allowscientists to share workflows, which will help to createa collaborative research space to help advance the stateof the art in this area. Such an environment is beingdeveloped through the WHIP5 project, based aroundtools developed in the myExperiment project [8].

In order to distribute such workflows, a complex up-loading, packaging, and deployment subsystem mustexist; not only to bundle the XML document speci-fying the workflow along with all the tools, resourcesand Java class files for execution on the remote plat-form, but also to be able to fan-out these bundles po-tentially to millions of peers. For this to be achieved,we cannot rely on the current, centralised mechanismemployed for data distribution in BOINC6 (BerkeleyOpen Infrastructure for Network Computing, an open-source software platform for computing using volun-teered resources) as this is not only expensive in bothcost (i.e. we would need to buy a fairly heavyweightserver for this purpose) and time (for administration),but it would also be extremely slow for updating theworkflows to the network participants as existing Tri-ana toolboxes for the analysis of such audio are over 10megabytes in size. Further, such latency is simply notacceptable for the scientists who would like to be ableto prototype their ideas quickly and not have to wait afew days for everyone to be updated before they canview the results.

To address this problem, we have chosen a peer-to-peer approach that is based on the super peer architec-ture but extends this idea to employ the use of securedata servers (called package repositories) that cache theworkflow bundles for DART, to be able to replicate anddecentralise the distribution of the workflows. Othertechniques were considered, such as bittorrent, but this

5http://www.whipplugin.org/.6see BOINC website http://boinc.berkeley.edu/.

is unacceptable within the BOINC framework becauseof security constraints. First, it requires every user onthe network to open a port for serving the data and sec-ondly, it provides no scoping environment for DARTto be able to restrict which servers can act as a dataprovider for their workflows. We would like scientiststo be able to develop their ideas in confidence (and alsoto support commercial products) and therefore a usermust be able to impose control on who is authorisedto distribute the data. For this we employ the use ofX.509 certificates that are issued by a Certificate Au-thority (CA), in this case the Dart manager, and madeavailable to certain participants for authentication tobecome package repositories.

DART is a realisation of a Cardiff University researchframework called Alchemist, which provides the un-derlying P2P infrastructure and mechanisms to createsecure data caching groups for implementing the dis-tributed package repositories. This framework is alsoused by another project that implements data distribu-tion techniques for scientific data. More informationabout this framework can be found in publications [3,14]. The scheme described here for distributing work-flows therefore has many applications in distributedsystems as a whole.

The rest of the paper is organised as follows; in thenext section we discuss some of the background tech-nologies and related work within this paper, includinga brief overview of Triana and a workflow example.In Section 3, we describe the DART framework itself,followed by a discussion of our simulation results forthe application and network in Section 4, which showshow the system will scale as the number of peers (users)in the network increases. Finally the conclusion willargue that this approach to distributing dynamic com-putational workflows is valid, and outline the futurework.

2. Background technologies

DART represents a fusion of recent Internet scaledistributed systems technologies. It encompasses se-curity techniques from Grid computing, which hasevolved from toolkits such as Globus7 that provide thecore mechanisms for managing the execution of jobsand transfer of data. The Alchemist framework, onwhich DART is built, is also based on Web services

7http://www.globus.org/.

E. Al-Shakarchi et al. / Distributing workflows over a ubiquitous P2P network 271

with support for the Open Grid Services Architecture(OGSA) [7] approach to expose resources using Webservices standards8 within a service oriented architec-ture (SOA). The recent implementation of OGSA in theform of WS-RF [6] provides decoupled mechanismsfor representing stateful resource capabilities throughstateless Web services interfaces, which allow a systemto manage lifetime without using a tightly coupled ap-proach such as those used in previous distributed objectsystems, e.g. Corba9 or Jini.10 Grid computing is builton standardised technologies and extended through oth-er standardisation efforts, often initiated through thevarious research and working groups hosted within theOpen Grid Forum11 (OGF). Both Alchemist, and there-fore DART, intend to adhere to these standards as theyevolve.

DART also employs the use of Peer-to-peer (P2P)technologies, which have grown through the develop-ment of popular applications targeted at specific ser-vices, such as Napster,12 Gnutella,13 and CPU shar-ing systems like SETI@home, with no desired path totheir interoperability or standardisation. P2P design-ers have been more concerned about solving practi-cal problems, such as discovery and communicationbetween huge numbers of unreliable edge peers, anddeveloping scalable and robust mechanisms to tacklefrequently disconnecting peers and unfriendly networkapplication environments, e.g. Network Address Trans-lation (NAT) systems and firewalls. More often, thesociety of peers cannot be trusted and therefore the con-tent of a file or service cannot be the guaranteed. Thedata distribution techniques employed here build uponthe super peer network structure that has been shownto cope with highly transient participants on a massivenetwork scale.

Thirdly, DART builds on tools within the workflowdomain for e-Science applications, where there has re-cently been a wealth of activity from specific applica-tion domains, such as the support for scientists to con-duct in silico experiments in biology using Taverna [13]to more generalised systems, such as Kepler [12] orthe VDS [4]. For a state-of-the-art overview of currentworkflow environments and toolkits, see [15].

Audio analysis algorithms and frameworks for Mu-sic Information Retrieval (MIR) are expanding rapidly,

8http://www.w3.org/TR/ws-arch/.9http://www.omg.org/.10http://www.jini.org/.11http://www.ogf.org/.12http://www.napster.com.13http://www.gnutella.com/.

providing new ways to garner information from audiosources well beyond what can be ascertained from ID3tags, the standard format for storing artist and songmeta-data within an MP3 audio file. Modest success-es have been made in audio-based musical genre clas-sification audio-analysis algorithms such as musicalgenre classification [1,18], beat detection and analy-sis [5], similarity retrieval [1,11,21], and audio finger-printing [9]. This work uses Short-Time Fourier Trans-forms to track the means and variances of the SpectralCentroid, standard deviations of the spectrum aroundits centroid, spectral envelopes, and signal power torepresent sound textures, beat and pitch content [17].These values are then transformed into attribute–valuepairs for pattern matching and semantic retrieval. Thereis still much to be done in this field. Refinements toexisting strategies, as well as new strategies are stillneeded.

The analysis component of MIR requires extensivecomputational resources. Distributed environmentsand P2P networks are already being used for this pur-pose [19]. The idea of using MIR over P2P was pro-posed in Wang et al. [20], however this system sufferedfrom problems with scalability. More recently, the JX-TA programming framework was used by Baumann [2]to aid in the content-based retrieval over a P2P network.The proposed DART system differs from the distribut-ed MIR system proposed in [19] however, in that onlymetadata is returned to the main Triana server for anal-ysis, as opposed to actual audio data files, and has adifferent overall goal; DART is not intended to act as afile sharing system, but instead a distributed P2P MIRsystem with the main application scenario focussing onthe recommendation of music based on the audio filesalready stored on the user’s hard drive.

Pandora14 and is a novel music recommendationsys-tem from the makers of the Music Genome Project.15

Pandora allows users to enter the names of artists orsongs they like, and Pandora will consult return a playlist of artists and songs that the user may like. DARThopes to build on these concepts by using the user’sactual audio files, extracting information based on thecharacteristics of the audio, and using these attributesto base the recommendations on

Last.fm16 is an internet radio and music com-munity website, and uses an MRS known as an

14http://www.pandora.com/.15http://www.pandora.com/mgp.shtml.16http://www.last.fm/.


Audioscrobbler.17 Last.fm builds a profile of eachusers’ musical taste by recording details of all the songsthe user listens to using their media player (iTunes,Winamp, etc) or iPod. This information is ‘scrobbled’(transferred to Last.fm’s central database) via a plugininstalled into the users’ music player, and the users’profile data is displayed on a personal web page. TheLast.fm website offers numerous social networking fea-tures and can recommend and play artists similar tothe user’s favourites. Again, DART hopes to build onthis by performing audio analysis in addition to rely-ing on statistical data, and will also employ a decen-tralised distributed P2P model. However, the Last.fmAudioscrobbler system also exposes its data via web-services so other projects can make interesting use ofthe data and statistical results and recommendations inthe database. The feasibility of making use of this sta-tistical data in DART in addition to the audio analysisworkflows, is currently being researched by the DARTteam.

In summary, DART’s P2P architecture aims to buildupon all of these developments to provide an advanced,fully scalable platform for developing, testing and de-ploying new search and analysis algorithms on an In-ternet scale. Furthermore, as explained later in the pa-per, the DART system can be adapted to fulfil a vari-ety of applications other than music recommendationby modifying the Triana workflow that is distributed tothe worker nodes. The Triana framework and Trianaworkflows are discussed in the next section.

2.1. The Triana workflow environment

Triana18 is a graphical Problem Solving Environ-ment (PSE) for composing data-driven applicationsby executing workflows. Workflows (data or control-flows) are constructed by dragging programming com-ponents, called tools or units, from the toolbox ontoa workspace, and then drawing cables between thesecomponents to create a graph. Components can be ag-gregated visually to group functionality and composenew algorithms from existing components. For ex-ample, to add a digital Schroeder reverb to a piece ofaudio (see Fig. 1), the file could be loaded (using theLoadSound unit), then passed to a SchroderVerb groupunit before being passed to the Play unit to hear theresult. The SchroderVerb unit is itself a group, which

17http://www.audioscrobbler.net.18http://www.trianacode.org.

consists of a number of summed comb delays and all-pass filters, representing the inner workings of such analgorithm.

Within Triana, a large suite of Java tools exist in arange of domains including signal, image and text pro-cessing. The most advanced tools in Triana are createdfor signal processing, as Triana was initially developedfor signal analysis within the GEO600 GravitationalWave Project,19 who use the system to visualize andanalyse one-dimensional signals (rather like an audiochannel but sampled at a lower rate). Therefore a num-ber of core mathematical, statistical and high-qualitydigital-signal processing algorithms already exist.

Triana integrates Grid, Web service, and P2P tech-nologies, and has been used in a number of domains,from bioinformatics, investigating biodiversity pat-terns, to gravitational wave observation, using compu-tational Grids to process one-dimensional signals usingstandard digital signal-processing (DSP) techniques.The goal of the DART project is to leverage this tech-nology such that the same kind of DSP processing canbe achieved with audio rate signals for the purposes ofsignal analysis, feature extraction, synthesis, and musicinformation retrieval.

Given its modularity, its support for high qualityaudio, and its ability to distribute processes across aGrid of computers, Triana has the potential to be anextremely useful piece of software that allows users toimplement custom audio processing algorithms fromtheir constituent elements, no matter their computation-al complexity.

Triana’s data type classes are fundamental to Tri-ana’s flexibility and power. Data types are containersfor the data being processed by the units or tools. Thetwo main types in Triana that are relevant to the pro-cessing of digital audio in Triana are MultipleChannel,a base class for representing multiple channelled data.The MutipleAudio data type stores many channels ofsampled data, again, each channel can have its ownparticular audio format of the data e.g. the encodingformat, sample rate and number of bits used to recordthe data. These data types are essential for readingand writing sound files, and analysing audio data. Inessence, MultipleAudio provides the support for highquality audio to be processed within Triana.

The Triana audio toolkit consists of several cate-gorized hierarchical folders, each with an assortmentof units based on the MultipleAudio Triana data type.

19http://www.geo600.uni-hannover.de/.


Fig. 1. A simple audio processing work-flow, showing how a Schroeder reverb is applied to a signal by using a group, which contains theunderlying algorithmic details.

This type utilises the JavaSound API classes in order toallow the use of high fidelity audio. The Audio toolkittree is split into three main folders: Input, Output, andProcessing.

The sub-categories inside these folders should berecognizable to regular users of audio processing orproduction software, with sub-folders such as Convert-ers, Delay, Dynamics, Modulation, MIR, and EQ units,to name a few. Each folder contains units which arein themselves effects but also allow the user to cre-ate their custom algorithms from the smaller buildingblocks supplied. This unit aggregation gives the usermore freedom to take advantage of Trianas modularity.

One feature of interest resides in the Converters fold-er; two units are available that allow the user to convertfrom MultipleAudio to a SampleSet Triana data type –and back again (MAudioToSSet and SSetToMAudiorespectively). This opens up a whole range of possibil-ities to the user, enabling them to utilise many of thenumerous math and signal-processing units that workwith the SampleSet data type) to process the audio, andthen convert data back to a MultipleAudio data type forplayback. One example of how this technique could beused is shown in Fig. 2. In this example, a Stereo2Monounit is used to split the stereo channels of an audio file

or stream into two distinct mono channels. Each side isthen converted to a SampleSet and fed into a Subtractorunit from the Math folder. This subtracts the left fromthe right stereo channel, which results in the removalof sound that is contained in both i.e. those panned inthe middle of the stereo field. This is a simple wayof removing vocals from many songs and leaving the(majority) of the backing track (as vocals in particularare normally panned down the centre). This is just asimple example of how a few of the converter unitscould help users create their own algorithms.

As mentioned previously, Triana also contains hun-dreds of statistical, mathematical, and signal process-ing units, which can be used in conjunction with all ofthe MultipleAudio compatible units, opening up an vastrange of units to facilitate and aid MIR and the creationof useful MIR algorithms. Triana includes Fast FourierTransform units, a range of filters, graph and histogramviewers, spectrum analysers and more, meaning thatscientists can take advantage of pre-written softwaremodules within Triana to aid in the development of newalgorithms. This allows the user to bypass the con-ventional approach to programming; creating variousmethods and coding a core program to connect the setof related method procedures together.


Fig. 2. Splitting stereo audio and subtracting the left from right channel.

Triana is capable of working within P2P environ-ments through the use of P2PS and JXTA and it canwork within Grid environments through the use of theGlobus toolkit accessed via the GAT/GAP interface.Further, it has the capability of fusing these environ-ments through the use of WSPeer, which can host WebServices and OGSA implementations, such as WS-RF,within P2P environments like P2PS, see [10,16] for afull description.

In DART, we are interested in forming unstructuredP2P networks and therefore need to employ technolo-gies that can adapt and scale within such an environ-ment. For distribution across dynamic networks, weuse the P2PS binding for Triana. Peer-to-Peer Sim-plified (P2PS) was a response to the complexity andoverhead associated with JXTA. As its name suggests,it is a simple yet generic API for developing P2P sys-tems. P2PS encompasses intelligent discovery mech-anisms, pipe based communication and makes it pos-sible to easily create desirable network topologies forsearching, such as decentralised ad-hoc networks withsuper peers or rendezvous nodes. P2PS is designed tocope with rapidly changing environments, where peersmay come and go at frequent intervals.

At the core of P2PS is the notion of a pipe: a virtualcommunication channel that is only bound to specific

endpoints at connection time. When a peer publishesa pipe advertisement it only identifies the pipe by itsname, ID, and the ID of its host peer. A remote peerwishing to connect to a pipe must query an endpointresolver for the host peer in order to determine an actualendpoint address that it can contact. In P2PS a peercan have multiple endpoint resolvers (e.g. TCP, UDPetc), with each resolving endpoints in different trans-port protocols or returning relay endpoints that bridgebetween protocols (e.g. to traverse a firewall). Also,the P2PS infrastructure employs XML in its discoveryand communication protocols, which allows it to beindependent of any implementation language and com-puting hardware. Assuming that suitable P2PS imple-mentations exist, it should be possible to form a P2PSnetwork that includes everything from super-computerpeers to PDA peers.

In the DART framework the distribution policy forTriana is loosely coupled. Although Triana acts asa manager and processor in the system, the distribut-ed functionality is provided by the DART framework,which implements a decentralised discovery and com-munication system based on P2PS. This allows Trianaworkflows to be uploaded to peers for execution and en-ables users to query the network to locate results and to


perform custom searches. The DART system thereforemanages the specifics of the network and Triana actsas a client (i.e. the DART manager or user) that bothaccesses DART and also acts as an end processor toexecute workflows that have been previously uploadedfor the analysis of the audio. Therefore, Triana doesnot tie the network together, rather it accesses a looselycoupled framework that allows wide-range distributedTriana entities to communicate via the Internet. Sec-tion 3 of this paper discusses the DART framework inmore detail and details the mechanisms to facilitate so-called work package assignment and the retrieval of theresults.

3. Dart: A framework for distributed informationretrieval

3.1. Work package assignment and results retrieval

Many DART applications are process-intensive andcan require the distributed analysis of a large number ofaudio files in order to provide some useful, non-trivialfeedback to the user. One of the main aims of the DARTsystem is the development of a music recommendationsystem, whereby music on a user’s hard drive (usuallyin MP3 format) is analysed, and recommendations aremade to the user based on the results of the analysis.Both index based analysis, including statistical corre-lations between song names to extract commonalitiesin order to make a recommendation, and audio contentbased analysis such as searching the track for tempos,timbre, mood, pitch, and frequency range etc., can beused to analyse the audio. Algorithms for both types ofanalysis are composed in Triana (as discussed earlier),from workflows that are created by the DART Manager.These workflows are bundled into a workflow “pack-age” (which can also be called a “DART package,” andcontains the Java code and their required resources) thatis distributed onto the network in order to upload thenew analysis to the users (or workers) machines. Asthe algorithms are updated and refined by the DARTManager, the updated Triana workflow and any newtools are bundled into a new package, and propagatedonto the network.

The scenario discussed in this section describes thedecentralised DART network, in which nodes are or-ganized in a super peer topology using the P2PS mid-dleware. In P2PS, producers (e.g. providers of pack-ages containing workflows or results) create adverts toadvertise that they have something that is of interest to

other participants in the network. Consumers (i.e. thepeers that wish to use available packages, result setsand so on) issue queries in order to search for rele-vant adverts. P2PS rendezvous nodes are then respon-sible for matching queries with adverts within their lo-cal cache, in order to search for matches and respondappropriately. Consumers receive adverts when theirquery matches, and these adverts can be used to re-trieve the relevant information they require, i.e. down-load the new workflow package to perform the analysis.The DART Manager node produces and advertises theworkflow package representing the new DART bundle(called DART Package Adverts) containing algorithmsthat the worker nodes need to run (new Triana unitsand workflows). With the DART system, the data filesthat undergo analysis are on the users local systemshard drive, and therefore all data processing is local sonetwork bandwidth is not consumed transferring largedata files over the DART network. Although local, theprocessing is massively parallel, as participants analysetheir own audio files in parallel.

Simple peers (or “workers”) are available to exe-cute the algorithms and workflows, and therefore issuea package query to download a package in order tostart the analysis of their music collections. The entireworkflow is executed by each worker which downloadsthe package; the workflow is not then broken downand farmed out to separate nodes to complete differenttasks.

Super peer interconnections are used to make pack-age queries travel across the network rapidly; superpeers play the role of rendezvous nodes, since theycan store package adverts and compare these files withqueries issued to discover them; thereby acting as ameeting place for both package providers and con-sumers. Since packages could require a reasonableamount of storage space, it is assumed that only someof the peers in the network will cache these files. Thesepeers are called Package Repositories (PR) and can al-so be super peers or worker peers. Each node in thesystem decides if they want to be a super peer and/orpackage repository, as well as a worker.

Figure 3 shows a sample topology with 5 super peers(2 of which are also package repositories), and the se-quence of messages exchanged among different nodesin order to perform the package submission protocol.These messages are related to the execution of a work-flow by a single worker, labelled as W0. Note that inthis figure, normal peers are not considered as packagerepositories.

When a new DART workflow package is avail-able, the DART manager puts this package on one or


Super Peer

Package Repository Worker and User

DART Manager

1,3

1,31,3

2

1

4

4

w0

3

5

Messages

1. PackageQuery2. PackageAdvert3. DataQuery4. DataAdvert5. Download Request

Fig. 3. Super peer protocol for the dissemination of workflow pack-ages: sample network topology and sequence of exchanged messagesto execute one package cycle.

more package repositories and propagates an associat-ed metadata file, or package advert, on the super peernetwork. This advert is an XML file describing theproperties of the algorithms to be executed (i.e. work-flow parameters containing the units/tools, platform re-quirements if any, information about required input au-dio data files, etc.).

When available to offer some of its CPU time, aworker searches the network to verify if a new ver-sion of the package is available. More specifically, theworker sends a package query that travels the networkthrough the super peer interconnections (message 1 inFig. 3). A package query is expressed by an XMLdocument that contains hardware and software featuresof the worker node, if this is necessary e.g. availableRAM, disk space or JDK version. The query succeedswhenever it matches an advert of a package that canbe actually executed by the requesting worker. Thispackage advert is then sent directly to the worker.

Thereafter, the worker must search for a packagerepository that stores the updated workflow package,and sends a data query to the network. As more thanone package repository can match the query,a matchingrepository does not send the package directly to theworker, in order to avoid multiple transmissions of the

same file. Conversely, the repository returns only adata advert to the worker.

A worker can choose a repository according to poli-cies that can rely on the distance of repositories, theiravailable bandwidth etc. Then the worker initiates thedownload operation from the selected repository.

The DART protocol allows for the progressive dis-semination of workflow packages on different repos-itories. Initially these packages are stored on one orfew repositories. However, when a worker downloadsa package, if the local super peer plays also the role of arepository, the package is first downloaded and cachedby this super peer, then forwarded to the worker. In thefuture another package query can be matched directlyby this package repository. Replication of the work-flow package on multiple package repositories allowsfor a significant saving of time in the querying phaseand enables the simultaneous retrieval of packages fromdifferent repositories.

Once the worker has received the updated package,the workflow is executed by the workers, and they be-gin to analyse the audio files on the workers systemduring the systems idle time. Once a package cycleis complete and the worker has results to present, theworker then creates an XML advert containing the re-sults and metadata generated by the algorithm specifiedin the package (a results advertisement). As the actualresults generated would be extremely small in size inthis DART system, the functionality of the super peerhas been extended in order to also cache and make themavailable.

Each worker on the network can also be thought ofa results provider on the DART system, as well as actas a user, as it can query for results (in this case a suit-able music/song suggestion as generated on the superpeer). There is no central results collector, but ratherDART utilises a fully decentralised model and allowsthe results to propagate through the network hop-by-hop, to be stored on the super peers. The super peerscan process the metadata and issue an XML results ad-vertisement on receipt of a results query from the user.Once the query is received, the results may be sent tothe user.

3.2. Peer overview

This section gives a brief overview of what role eachnode on the network plays, and the jobs associated withthat role:


DART Manager

P2PSNetwork

User

New Packages

Super PeersPackage Repositories

Super Peers Package Repositories

Workers

Workers

WorkerSong Suggestions

ResultAdvertisements

Consumer

Audio Analyser/Provider

Fig. 4. High-level overview of the DART system, showing the various peers and their connectivity.

DART manager– Creates new workflows and Triana units– Advertises new DART Packages

User– Queries Results– Downloads/Receives Results

Worker Nodes– Queries New Packages– Downloads Package– Works/processes Workflows– Advertises Results

Package Repositories– Query and Download New Packages (Stored lo-

cally)– Advertise Packages

Super Peers– Caches Advertisements– Performs simple analysis of results received from

workers

3.3. Workflow design

The workflow created by the DART manager and dis-tributed to the other peers on the network, will alwaysbe evolving. Initially, it is easier to concentrate on sta-tistical methods for recommending music to the usersof the DART system, however this will soon evolve tobecome more involved as the Triana workflow is re-fined and more MIR algorithms are incorporated in-to the package sent to the worker nodes. The algo-rithms used for the analysis and recommendation ofmusic will be made available to the MIR communityfor refinement, suggestions, and also to allow for theadvancement of this field of research as MIR-algorithmspecialists provide input ideas and offer improvementsto both refine and maximise the benefit of the use ofthe DART system.

3.4. DART flexibility

So far this paper has focussed on the topic of dis-tributing Triana workflows over a ubiquitous P2P net-


work in order to facilitate the recommendation of musicto its users, and the DART framework created in orderto achieve this. However the DART infrastructure canbe used to distribute workflows for a variety of applica-tions, musical and otherwise. One application scenariois a DART system installed on a local/closed network;scanning sound effect audio files on separate systemsin networked commercial studio facilities, to search forsound effect files (or for example, drum sounds) thatmatch a specific criteria. For example, if a DART userrequires a snare drum sample, usual search mechanismscannot search for suitable sounds unless the audio filesfilename contains the word “snare” or other suitableidentifier – the search mechanism would do no morethan string matching.

However, given the appropriate workflow created bythe DART manager, when propagated onto the networkthe DART system would allow the user to search allthe audio content in the distributed sound library, andsuggest files with suitable characteristics that match thespecific search criteria set by the user. This would alsobe able to return further results, which would not bereturned via conventional methods – any samples thatmatch the users search criteria, will be returned. Thismeans that when ignoring criteria such as pitch andtempo and investigating the timbre of the sound file,new sounds that could be adapted to work for the user,could be suggested. For example, a sped up sample ofa car collision may easily work well in place of a snaresample!

Another example consists of a scenario that requiresspeech recognition algorithms to detect a certain callerfrom a large amount of recorded telephone audio datathat was distributed over several machines. Given asophisticated enough algorithm (created in Triana as aworkflow), the DART framework could be used to scanterabytes of recorded audio data to help trace calls froma specific caller. The flexibility of the DART systemstems from the ability to easily refine and change theTriana workflow that dictates what the worker nodeswill be processing in their screensaver/idle/nice time.

4. Distributed simulations

In order to demonstrate the DART system’s poten-tial to scale and distribute workflows over a ubiquitousP2P network as the number of participants increase,ICAR-CNR (The Institute of High Performance Com-puting and Networking) in Italy, and Cardiff Univer-sity, Wales, conducted a study in which a simulation

of the DART scenario (as discussed in Section 3) wasrun. A simulation analysis was performed by means ofan ad hoc event-based simulator, written in C++, toevaluate the performance of the package disseminationprotocol described in Section 3.1.

The simulation scenario is described in Table 1. Inthis case, a workflow package of around 10.4 mb insize (the current size of the Triana audio toolkit) is tobe distributed to the worker nodes on the network. Itis here assumed that the workflow can be executed byany worker and that only super peers can be packagerepositories.

This scenario simulates the performance and be-haviour of a distributed P2P network with 1,000 to20,000 workers, with a maximum value of 2,000 superpeers as the number of super peers is assumed to be10% of the number of workers. Workers can disconnectand reconnect to the network at any time. This impliesthat the download or execution of a workflow fails uponthe disconnection of the corresponding worker. It isassumed that connections between two adjacent superpeers have a larger bandwidth and a longer latency thanlocal connections (i.e. between a super peer and a localsimple node).

In the simulation scenario, each worker has to down-load and execute a workflow package; to this aim itissues a package query and follows the protocol de-scribed in Section 3.1. If the download operation failsdue to a worker disconnection, a new package query isforwarded and the procedure is repeated.

The experiment is aimed at evaluating the effective-ness of the dynamic caching mechanism described inSection 3.1. Therefore, the number of available pack-age repositories was varied from 1 to half the numberof super peers: one of these repositories provides theworkflow package from the beginning, while the othersact as cachers, as they can download, store and providedata on the fly.

Simulations have been performed to analyze theoverall dissemination time, Tdiss, defined as the timeneeded to propagate the workflow package to at least95% of the workers. This time is crucial to determinethe rate at which workflow packages can be retrievedfrom the package repositories in order to guarantee thatthe workers are able to keep the pace with the availabil-ity of new versions of the package. The average timeneeded to perform a single download operation, T dl, isalso calculated.

The average utilization index of package reposito-ries, Pact, is defined as the fraction of time that a pack-age repository is actually utilized, i.e., the fraction of


Table 1Simulation scenario

Scenario feature Value

Number of workers, or simple peers, Npeer 1,000 to 20,000Number of super peers, Nspeer 100 to 2,000Average number of workers connected to a super peer 10Maximum number of neighbors for a super peer 4Average connection time of workers 4 hoursAverage disconnection time of workers 1 hourNumber of package repositories 1 to 50% of Nspeer

Size of input data files 10.4 MbytesLatency between two adjacent super peers 10 msLatency between a super peer and a local worker 10 msBandwidth between two adjacent super peers 2 MbpsBandwidth between a super peer and a local worker 1 MbpsMean workflow execution time 10 hours

1e+03

2e+03

5e+03

1e+04

2e+04

5e+04

1e+05

0 100 200 300 400 500 600 700 800 900 1000

Td

iss

(s)

Number of package repositories

Npeer = 1000Npeer = 5000

Npeer = 10000Npeer = 20000

Fig. 5. Time at which 95% of workers have downloaded a newversion of the workflow package from a package repository.

time in which at least one download connection, froma worker (or another repository), is active with thisrepository. The value of Pact is averaged on all therepositories and can be seen as an efficiency index.

Figure 5 shows that the dissemination time decreas-es as the number of package repositories increases,as worker nodes can exploit higher parallelism anddownload workflow packages from multiple reposito-ries (possibly closer) and also because the reposito-ries themselves are under less stress and therefore datadownload time decreases. Conversely, if the numberof repositories is constant, the dissemination time in-creases with the number of workers as more downloadoperations must be performed and therefore a singlerepository has to serve more workers on average.

In these simulations, the protocol is shown as scal-able when one observes the results obtained with a fixedpercentage of repositories. As an example, observe re-sults obtained when the number of repositories is set to5% of peers (i.e., 50% of super peers, see dashed linein Fig. 5). It is interesting to note that as the number of

1e+03

2e+03

5e+03

1e+04

2e+04

0 100 200 300 400 500 600 700 800 900 1000

Mea

n d

ow

nlo

ad t

ime

Tdl

(s)



Npeer = 10000Npeer = 20000

Fig. 6. Average download time of single worker from package repos-itory when worker disconnection does not interrupt the download.

peers increases, the dissemination time increases veryslightly, much less than the number of peers. For ex-ample, with 1,000 peers and 50 repositories, dissemi-nation time is approximately 2,200 seconds; howeverwith 20,000 peers and 1,000 repositories,disseminationtime is approximately 4,100 seconds.

Figure 6 shows the average download time of a singleworker from a package repository when a worker dis-connection does not interrupt the download operation.The results here are analogous to the behaviour previ-ously observed when considering Fig. 6; the qualitativebehaviour is the same, but the values are lower. Thedownload time decreases as the number of repositoriesincreases, and the download time will increase as thenumber of workers increases.

Figure 7 shows the percentage of time in which arepository is actually exploited (i.e. there is at least onedownload in progress). We can observe that as morerepositories become available, the percentage of timedecreases; therefore, one should be hesitant in settingup a high number of repositories, because while this can


20

30

40

50

60

70

80

90

100

0 100 200 300 400 500 600 700 800 900 1000

Per

c. o

f P

R a

ctiv

ity t

ime

Pac

t



Npeer = 10000Npeer = 20000

Fig. 7. Percentage of time in which a package repository is actuallyexploited (at least one download in progress).

0

5

10

15

20

25

30

0 100 200 300 400 500 600 700 800 900 1000

Per

centa

ge

of

inte

rrupte

d d

ow

nlo

ads



Npeer = 10000Npeer = 20000

Fig. 8. Percentage of interrupted downloads.

slightly decrease dissemination time, they can also beunder-exploited. Another interesting result is that theutilisation of a given number of repositories increasesas the network becomes bigger and more workers needto download the workflow package. This is another ver-ification of the scalability behaviour discussed earlier.It should be noticed that this percentage never reaches100% because a data cacher (a package repository thathas no data at the beginning) must download data fromanother repository before it can serve a worker.

Figure 8 displays the percentage of download op-erations that are interrupted due to the disconnectionsof corresponding downloading workers. In this sim-ulation, only results for which this percentage is low-er than 30% are displayed. It was observed that thepercentage of interrupted downloads decreases as thenumber of repositories increases. In fact, the downloadtime decreases if more repositories are available (seeFig. 6), then a worker has more chances to conclude

its download operation. Finally, if the percentage ofrepositories with respect to the number of peers is set toa given value (for example 5%), the percentage of inter-rupted downloads is almost constant (see dashed line),which is a further confirmation about the scalability ofthe dissemination protocol.

5. Conclusions

The DART Music Recommendation System (MRS)project makes novel use of dynamic workflows in amassively distributed P2P environment. Remote pro-cessing on peers in the DART network is performedthrough the execution of workflows designed for MusicInformation Retrieval (MIR), and dynamically propa-gated through the network and discovered using a su-per peer mechanism. The scientist or analyst (DARTmanager) that creates the workflow is able to modify itand retransmit it to the peers, fine tuning the applicationbased on results.

We have performed extensive simulations using anad-hoc event simulator to explore how the transmittedworkflows will propagate throughout the network asthe number of peers and super peers increases. Theresults show that the network will scale as the numberof members increases as long as the number of superpeers that act as data providers increases by the sameratio. This shows that the real application should becapable of operating at an Internet scale with accept-able performance. Given the simulation results, a realnetwork and application is the next goal of the project.

References

[1] J.J. Aucouturier and F. Pachet, Representing musical genre: Astate of the art, Journal of New Music Research 32(1) (2003),83–93.

[2] S. Baumann, Music Similarity Analysis in a P2P Environment,Proceedings of the 4th European Workshop on Image Analysisfor Multimedia Interactive Services, London, UK, April, 2003.

[3] P. Cozza, C. Mastroianni, D. Talia and I. Taylor, A super-peerprotocol for multiple job submission on a grid, in: Euro-Par2006 Workshops, volume 4375 of LNCS, W. Lehner, N. Mey-er, A. Streit and C. Stewart, eds, Dresden, Germany, June2007. Springer Berlin/Heidelberg. ISBN: 978-3-540-72226-7,pp. 116–125.

[4] E. Deelman, C. Kesselman, G. Mehta, L. Meshkat, L. Pearl-man, K. Blackburn, P. Ehrens, A. Lazzarini, R. Williams andS. Koranda, GriPhyN and LIGO, building a virtual data gridfor gravitational wave scientists, HPDC (2002), 225–.

[5] J. Foote and M. Cooper, Audio retrieval by rhythmic similarity,in: Proceedings of the International Conference on MusicInformation Retrieval, volume Paris: IRCAM, 2002, pp. 265–266.


[6] I. Foster et al., Modeling Stateful Resource with Web Services,http://www.ibm.com/developerworks/library/ws-resource/ws-modelingresources.pdf.

[7] I. Foster, C. Kesselman, J. Nick and S. Tuecke, The Physi-ology of the Grid: An Open Grid Services Architecture forDistributed Systems Integration, Technical report, Open GridService Infrastructure WG, Global Grid Forum, 2002.

[8] C. Goble and D. De Roure, myExperiment: social networkingfor workflow-using e-scientists, Proceedings of the 2nd work-shop on Workflows in support of large-scale science, 2007,1–2.

[9] J. Haitsma and T. Kalker, A highly robust audio fingerprintingsystem, In Proceedings of the Third International Conferenceon Music Information Retrieval, volume Paris: IRCAM, 2002,107–115.

[10] A. Harrison, I. Wang, I. Taylor and M. Shields, WS-RF Work-flow in Triana, International Journal of High PerformanceComputing Applications (IJHPCA) Special Issue on WorkflowSystems in Grid Environments, to be published 2007.

[11] B. Logan and A. Salomon, A Content-Based Music SimilarityFunction, Cambridge Research Laboratory. Technical ReportSeries. Jun, 2001.

[12] B. Ludascher, I. Altintas, C. Berkley, D.G. Higgins, E. Jaeger,M. Jones, E.A. Lee and Y. Zhao, Scientific workflow man-agement and the kepler system. concurrency and computa-tion: Practice and experience, Concurrency and Computa-tion: Practice and Experience, Special Issue on ScientificWorkflows, 2006. to appear.

[13] T. Oinn, M. Greenwood, M. Addis, M.N. Alpdemir, J. Fer-ris, K. Glover, C. Goble, A. Goderis, D. Hull, D. Marvin,P. Li, P. Lord, M.R. Pocock, M. Senger, R. Stevens, A. Wipatand C. Wroe, Taverna: Lessons in creating a workflow

environment for the life sciences, Concurrency and Computa-tion: Practice and Experience, special issue on Grid Workflow,accepted for publication, 2006.

[14] M. Shields and I. Taylor, Alchemist: user driven searchingin ubiquitous networks, Proceedings of the 1st internationalworkshop on Advanced data processing in ubiquitous com-puting (ADPUC 2006), 2006.

[15] I. Taylor, E. Deelman, D. Gannon and M. Shields, eds, Work-flows for e-Science, Springer, New York, Secaucus, NJ, USA,2007.

[16] I. Taylor, M. Shields, I. Wang and A. Harrison, Visual GridWorkflow in Triana, Journal of Grid Computing, Special Edi-tion on Workflow, To be published, 2005.

[17] G. Tummarello, C. Morbidoni, P. Puliti and F. Piazza, Seman-tic audio hyperlinking: a multimedia-semantic web scenario,In Proceedings of the 1st International Conference on Auto-mated Production of Cross Media Content for Multi-channelDistribution, 2005, pp. 111–115.

[18] G. Tzanetakis and P. Cook, Music genre classification of audiosignals, IEEE Transactions on Speech and Audio Processing10(5) (2002), 293–302.

[19] G. Tzanetakis, J. Gao and P. Steenkiste, A scalable peer-to-peer system for music information retrieval, Computer MusicJournal 28(2) (Summer 2004 2004), 24–33.

[20] C. Wang, J. Li and S. Shi, An Approach to Content-BasedApproximate Query Processing in Peer-to-Peer Data Systems,Grid and Cooperative Computing, Second International Work-shopi (GCC 2003), Shanghai, China, December, 2003.

[21] C. Yang, Music Database Retrieval Based on Spectral Similar-ity, Proceedings of the 2nd Annual International Symposiumon Music Information Retrieval, 2001, 37–38.

Submit your manuscripts athttp://www.hindawi.com

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014


Distributed Sensor Networks


Advances in

FuzzySystems

Hindawi Publishing Corporationhttp://www.hindawi.com

Volume 2014


ReconfigurableComputing

Hindawi Publishing Corporation http://www.hindawi.com Volume 2014


Applied Computational Intelligence and Soft Computing

Advances in

Artificial Intelligence


Advances inSoftware EngineeringHindawi Publishing Corporationhttp://www.hindawi.com Volume 2014


Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications


Hindawi Publishing Corporation

http://www.hindawi.com Volume 2014

Advances in

Multimedia


Biomedical Imaging


ArtificialNeural Systems

Advances in


RoboticsJournal of



Computational Intelligence and Neuroscience

Industrial EngineeringJournal of


Modelling & Simulation in EngineeringHindawi Publishing Corporation http://www.hindawi.com Volume 2014

The Scientific World JournalHindawi Publishing Corporation http://www.hindawi.com Volume 2014


Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in


distributing workﬂows over a ubiquitous p2p...

Documents