Performance tests and comparison of SMOANotification with other reference implementations

of WS-Notification

Bartosz Bosak, Krzysztof Kurowski, Mariusz Mamonski, Ariel Oleksiak,Micha l Witkowski

Poznan Supercomputing and Networking CenterNoskowskiego 12/14, 61-704 Poznan, Poland

Abstract

Asynchronous messaging protocols are promising alternatives to popular syn-chronous communication routines in Web Services and Service-Oriented Archi-tectures (SOAs). The publish/subscribe notification pattern, which is based onone-way asynchronous messages, in many cases can drastically reduce the uti-lization of communication links among services in the Internet as well as improvethe efficiency of the existing Service-Oriented system. To deal with various com-munication scenarios in SOAs, and more importantly enable their integrationand interoperability, a family of WS-Notification industry standards has beenproposed. In this paper we present a successful reference implementation ofWS-Notification called SMOA Notification together with its both performanceand functional comparison to other commonly used notification systems in dis-tributed systems such as Grids or Clouds. We successfully performed variousexperiments to evaluate four main WS-Notification frameworks widely adoptedin Grid and Cloud environments, namely Globus Toolkit, WebSphere Appli-cation Server, Apache ServiceMix and SMOA Notification. Obtained resultsshowed that our solution outperforms other notification frameworks accordingto some relevant evaluation criteria. Moreover, SMOA Notification as an opensource platform can help Web Services developers to deal with sophisticatedand more advanced notification scenarios that can be adopted to existing SOAbased systems.

Keywords: WS-Notification, Grid, SOA, Web Services, publish/subscribepattern, asynchronous communication

1. Introduction

The robust notification was always a relevant issue in large-scale distributedIT systems such as Grids or Clouds. However, early Web Services providedrelatively simple notification messaging features based on remote procedurecalls (RPC), and still many existing systems use this technique. Typically,

Preprint submitted to Elsevier January 13, 2011

a Web Service sends a SOAP message back to a consumer over the samenetwork connection on which the request arrived. The communication patternin this case is fully synchronous, often called the pull-style message exchange.In fact, many SOA environments and Web Services available today in theInternet are still based on the synchronous communication pattern. However,in the future we expect more and more distributed systems using asynchronousmessages and publish/subscribe patterns as there are many advantages ofsuch notification approaches. A general idea of the notification messagingmay be expressed as a routing of unidirectional messages between notificationgenerators producing notifications (corresponding to a certain event) andnotification receivers. In practice, distributed IT systems can consist of manynotifications generators, receivers and include a set of routing components.The registration entities in publish/subscribe notification scenarios are usuallyreferred as subscriptions, and they may vary in their complexity. Componentsthat create subscriptions are typically named subscribers, notification gener-ators are often called publishers or producers, whereas notification receiversare called consumers [1]. Currently, there are two main specifications whichdefine basic rules for building standards-compliant notification systems for WebServices: WS-Eventing [2] and WS-Notification [3]. While the first one is stillunder the standard submission phase and it is rather a simple specification,the second one has been already approved by the OASIS standardizationbody. In this paper we focus on WS-Notification defining relevant notificationinteractions among involved components. The family of WS-Notificationstandards defines a number of advanced procedures and schemas for buildingpublish/subscribe systems, in particular based on the brokered notification.Unfortunately, many available WS-Notification reference implementations donot meet performance requirements, they are not fully open platforms, theydo not fully implement standard notification schemas, or they were integratedtightly with larger complex solutions, and thus they can not run separately [4].Therefore, one of the key motivations for our development was to design andimplement an advanced open notification system to satisfy mentioned needs.Today, our system called SMOA Notification offers highly efficient notificationcapabilities that can be integrated easily with third party Web Services andSOA environments. Therefore, as a proof of concept, in this paper we presentmany functional and performance tests to compare SMOA Notification withother well known notification solutions, such as Globus Toolkit, WebSphereApplication Server and Apache ServiceMix.

The rest of this paper is organized as follows. In Section 2 we briefly discussthe state of the art and related work. Section 3 presents all the main notificationfeatures and schemas defined in the WS-Notification standard. A comprehen-sive description of SMOA Notification architecture and its main notificationcapabilities are presented in Section 4. Then, in Section 5, we present comparefunctions and main features of four different WS-Notification compliant noti-fication frameworks. Performance tests and analysis of all tested systems arediscussed in Section 6. Some improvements and future extensions in SMOA

2

Notification are presented in Section 7. Section 8 concludes our research anddevelopment activities.

2. Related work

The idea of using publish/subscribe communication pattern has been knownfor decades, even if many existing Web Services based systems use the rela-tively easy synchronous message exchange. In fact, both WS-Notification andWS-Eventing specifications are based on well-known notification schemas whichwere slightly modified to fit to Web Services technologies. Before we discussvarious notification scenarios in Web Services, it is worth to mention othernotification solutions used IT systems, such as CORBA [5] and JMS [6]. Acomprehensive comparison of publish/subscribe patterns with other commonlyused notification schemes were presented in Patrick et al. [7]. However, in thispaper we focus on various notification scenarios and Web Services, as well asother solutions used in distributed systems, are not presented. A very interest-ing discussion around notification specifications was provided by [8]. Authorsdescribe in detail various notification concepts and standards evolved over thelast few years, and emphasized many benefits of WS-Notification. The WS-Notification standard itself is a foundation for several notification frameworks,such as Globus Toolkit [9], Apache Muse [10], Apache ServiceMix [11] and Web-Sphere Application Server [12]. Mentioned examples are the most known sys-tems offering various notification routines for Web Services, however there arealso other less popular WS-Notification implementations, e.g. WS-Messanger[13] which provides functionality of the unified notification broker for both WS-Notification and WS-Eventing standards, or a distributed notification brokerdescribed by Quiroz and Parashar [14]. To date, various attempts were alsomade to compare different approaches for the transport level communication inWeb Services, naturally used by the higher level SOAP message exchange pro-tocol. For example, many functional and performance analysis of Web Servicestoolkits were demonstrated in [15]. Noteworthy study of various communicationsetups using popular Web Services toolkits was provided also by Van Engelenand Gallivan [16]. Moreover, Kowalewski et al. [4] made an effort to compareperformance of basic notification functions in several frameworks implement-ing WS-Notification specification. Nevertheless, to the best of our knowledge,there is a lack of analysis and comprehensive comparisons of WS-Notificationreference implementations. In the next sections we present relevant solutionsfor advanced notifications in Web Services, measure their performance as wellas discuss their usability and applicability.

3. OASIS WebServices Notification 1.3

WS-Notification is a family of related documents that standardize the no-tification concept and topic-based information distribution in SOA and Web

3

Services based systems. There are three normative OASIS specifications: WS-BaseNotification, WS-BrokeredNotification and WS-Topics. In a nutshell, WS-BaseNotification standardizes the terminology, concepts, operations, WSDLand XML schemas needed to express basic roles involved in publishing andsubscribing for notification message exchange among Web Services. WS-BrokeredNotification describes specific objectives for expanded version of no-tification messages exchange with the indirect NotificationBroker service. TheWS-Topics document defines mechanisms to organize and categorize items forsubscriptions known as Topics.

A reader should note that a relatively simple Web Services notificationframework can be built based on WS-BaseNotification and WS-Topics speci-fications, without implementing schemas and scenarios described in the WS-BrokeredNotification document. Consequently, each notification generator pro-duces some types of notifications as a result of a certain Situation, and addition-ally exposes a NotificationProducer Web Service interface for creating Subscrip-tions. Subscribers interested in receiving a certain type of notifications, invokethe Subscribe method offered by the NotificationProducer that must be imple-mented by the notification source. Finally, notifications produced by notificationsources may be distributed over consumers specified in the Subscriptions. Nev-ertheless, notification frameworks implementing only WS-BaseNotification, forinstance Apache Muse or Globus Toolkit, have several limitations. One of thekey limitations is that in WS-BaseNotification sources of notifications have tobe known in advance and accessible for Subscribers. Therefore, the integrationof new components with a WS-BaseNotification framework can be difficult andtime-consuming, especially for large and dynamic IT systems.

Many existing SOA and Web Services based systems require notifica-tion approaches that are more elastic, general and interoperable. WS-BrokeredNotification introduces a new definition of an intermediary Noti-ficationBroker component that separates notification sources, called Pub-lishers, from Subscribers and NotificationConsumers. Consequently, WS-BrokeredNotification is more advanced, but it defines precisely notificationschemas that can meet requirements of many realistic scenarios. In WS-BrokeredNotification the broker plays a role of NotificationProducer for Sub-scribers and NotificationConsumer for Publishers. Importantly, this specifica-tion covers brokered notification scenarios, in which Subscribers do not have toknow in advance who is a Publisher of a certain type of notifications, only alocation of a NotificationBroker is required. Similarly, Publishers do not need tobe Web Service implementing the NotificationProducer interface, but they onlyneed to know where the broker is located. As semantics of notification messagesare much more important than certain localizations of notifications, brokerednotification frameworks are much more flexible notification solutions. In factthey allow an easy integration of distributed components with a notificationsubsystem. Finally, brokered notification frameworks help users to establishthe communication among involved entities more transparently and dynami-cally. Two possible notification architectures according to WS-Notification arepresented in Figure 1. The first diagram shows WS-BaseNotification scenario,

4

whereas the second one shows brokered notification scenario defined in WS-BrokeredNotification.

Subscriptions: ........

Subscriptions: ........

NotificationProducer

Su

bscrib

e

Notify

Subscriber(1)

Subscriber(1)

Subscriber(s)

NotificationConsumer(2)

NotificationConsumer(2)

Notification Consumer(s)

Situation

NotificationBroker

Subscriber(1)

Subscriber(1)

Publisher(s)

Base Notification

Brokered Notification

Subscriber(1)

Subscriber(1)

Subscriber(s)

NotificationConsumer(2)

NotificationConsumer(2)

Notification Consumer(s)

Su

bscrib

e

Notify

Notify

Situation(s)

SituationSituation(s)

Figure 1: Two main notification scenarios defined by WS-Notification: Base and Brokered

The third mentioned document taken from the WS-Notification family de-scribes precisely the mechanisms of Topics. In general, WS-Notification Topicis a base item used for categorization and selection of notifications. In orderto guarantee unique names for Topics they should be assigned to appropriateXML namespaces. TopicNamespace consists of a set of Topics that belong to thesame XML namespace. Nevertheless, Topics inside a TopicNamespace can beorganized in hierarchical structures. Thus, WS-Notification Topics introducesthe main XML Schema for building standardized TopicNamespaces. Moreover,the standard defines concepts of TopicExpression and TopicExpression Dialectwhich allow easier selections of certain Topics. Three types of dialects are de-fined, namely: Simple, Concrete and Full. While Simple and Concrete dialectsallow to select only a single Topic, the third Full dialect supports also a groupselection of Topics.

5

4. SMOA Notification overview

SMOA Notification is an open source implementation of the family of WS-Notification standards. It supports the topic-based publish/subscribe patternfor the asynchronous message exchange among Web Services and other involvedentities. The main architecture of our system is based on a highly efficient,extended version of the NotificationBroker managing all items participating innotification events. Today, SMOA Notification offers sophisticated notificationcapabilities, e.g. notification message filtering, and it was successfully integratedwith different communication protocols and various Web Services security mech-anisms. The modular architecture of SMOA Notification also provides a greatopportunity for developers to build new extensions and plugins to meet otherspecific requirements.

4.1. SMOA Notification architectureSMOA Notification implements all three specifications included in the OASIS

WS-Notification 1.3 standard. The following list summarizes all major conceptsdefined in three mentioned WS-Notification documents, which were adopted bythe SMOA Notification system:

• Situation - a certain occurrence, known by Publisher. Situation is a reasonof dispatching a Notification;

• Notification - an artifact of a Situation containing information about thatSituation. Notification is represented as an XML document, dispatchedby NotificationProducer or Publisher, and sent to NotificationConsumer;

• Publisher - a unit that creates Notifications, detects Situations, and trans-lates it to Notification messages;

• Subscription - represents relation between Publisher and NotificationCon-sumer. Subscription is created by Subscriber and it contains filteringparameters, such as a Topic or content filters;

• Subscriber - a unit that creates Subscriptions. It sends the Sub-scribeRequest message to the NotificationProducer interface;

• NotificationBroker - an intermediary Web Service that decouples Pub-lishers and NotificationConsumers. It redistributes Notifications receivedfrom Publishers to NotificationConsumers basing on parameters of Sub-scriptions;

• NotificationConsumer - an endpoint receiving Notifications produced byPublishers and redistributed by NotificationBroker;

• PullPoint - a special entity that accumulates Notifications and offers an in-terface for retrieving these Notifications by remote NotificationConsumers.As PullPoint implements the NotificationConsumer interface it can be

6

provided in a Subscription as a receiver of Notifications - NotificationCon-sumer. In general, the main purpose of PullPoints is to enable pull-stylenotifications.

SMOA Notification supports also many topic-based and filtering WS-Notification features listed below:

• Topic - a concept used to categorize and filter Notifications. Topicsare typically used to determine which subscribed NotificationConsumersshould receive a specific Notifications;

• Topic Tree - a hierarchical group of Topics;

• TopicNamespace - a set of Topic Trees sharing the common namespace;

• TopicExpression - an expression for selecting Topics from TopicNames-pace;

• Dialect - identifies the type of a grammar used in TopicExpression. Thespecification defines three types of grammars (Dialects): Simple, Concreteand Full;

• MessageContent Filter - an additional type of filters used in Subscriptions.MessageContent Filters operate on a content of Notification messages todetermine whether a certain Notification should be forwarded to Notifica-tionConsumer.

SMOA Notification is a fully configurable notification framework. Fromthe architecture perspective, it exposes the well-defined Web Service interfacecorresponding to the role of NotificationBroker as well as it offers other standardinterfaces for managing Subscriptions and PullPoints. A general diagram ofthe system architecture is presented in Figure 2. The low-level architecture ofSMOA Notification is based on highly efficient mechanisms and data structures,e.g. hash tables that minimize overheads related to notification forwarding. Allnotification objects, including Subscriptions, registered entities, and relevantconfiguration settings can be also stored in the external database connected tothe system over efficient ODBC mechanisms.

As it is presented in Figure 2, all funtions offered by SMOA Notifica-tion can be invoked using different transport protocols. Currently, our sys-tem supports the SOAP communication over the following transport channels:HTTP, HTTPS, and additionally Extensible Messaging and Presence Protocol(XMPP), described for instance by Saint-Andre et al. [17]. Please note, thatthe high-level SOAP based notifications among Web Services can be transportagnostic. In other words, Subscriptions and PullPoints can be created and man-aged over different transport protocols, including mentioned HTTP/HTTPS,and XMPP. Thus, SOAP notifications can be sent by Publishers using XMPPand then received by NotificationConsumers via HTTP/HTTPS or vice versathanks to brokered notification capability implemented in SMOA Notification.

7

HTTP HTTPS XMPP

SOAP

PostgreSQL

ODBC

...

XML

XML Schema

...SecPalCerts

SMOA Core functions, libraries and plugins

PUBLISHER SUBSCRIBER

ADMINISTRATOR

NOT. CONSUMER

Topics PullPoints

PublisherRegistrations

Subscriptions

Config

Policy

SMOA Notification

Figure 2: SMOA Notification architecture

Finally, both management and administration of SMOA Notification are alsovery intuitive as various configuration parameters can be adjust to specific usersrequirements. Among other interesting options, it is very easy to tune the ser-vice to use or not to use a database, security or select certain transport protocolsby editing well-organized XML configuration file.

Advanced mechanisms for Subscriptions available in SMOA Notification pro-vide powerful filtering options. It is possible to create advanced filters basedon both Topics and the content of notification messages. To explain differ-ences between these two filtering mechanisms defined in WS-Notification, Table1 presents their brief comparison taking into consideration two phases of thefiltering lifecycle: creation and operation. Generally speaking, topic filteringis more efficient in the operation phase than the message content filtering, butit is less flexible and may be insufficient in very dynamic scenarios. In SMOANotification, subscribers can choose filtering mechanisms which will be properfor their needs.

4.2. Implementation detailsFrom the software development perspective, SMOA Notification is an in-

tegrated part of a bigger open source initiative that aims to provide a set ofdevelopment kits, communication tools and highly efficient Web Services tar-geted, but not limited, to high performance computing, grids and SOA environ-ments [18]. SMOA Notification is built upon the SMOA Core generic librarycommonly used also by other SMOA Web Services. The library offers a setof standard functions that simplify the development of any Web Service in C

8

Table 1: Topic and MessageContent filters comparison

Creation Operation

Topic Fil-ters

Subscriber defines in theSubscribeRequest messagethe TopicExpression in theone of the three possible Di-alects. The TopicExpressionselects a Topic or a set ofTopics that the Subscriptionis for and creates associ-ations from these Topicsdirectly to the Subscription.

Publisher specifies in a No-tify message a TopicExpres-sion that match exactly oneTopic. SMOA Notificationquickly looks up for thisTopic and the linked Sub-scriptions. Depending onthe configuration of SMOANotification, the Notify mes-sage may be now forwardedto Notification Consumersdefined in Subscriptions or,when the MessageContentfiltering is used, the messagecan be firstly checked by theMessageContent Filters.

MessageContentfilters

Subscriber defines in theSubscribeRequest messagethe MessageContent Filterexpression (e.g. a booleanXPath expression). TheMessageContent Filter ex-pression is assigned to theSubscription.

Notification message is ana-lyzed for satisfying the ex-pression specified in a Mes-sageContent Filter of a Sub-scription (e.g. XPath evalu-ation).

language. In particular, SMOA Core provides stable mechanisms for handlingWeb Services configuration, loadable modules, transport protocols, as well asauthentication and authorization. Moreover, it imposes some standard and bestpractices for developing new Web Services. Key features provided by SMOACore are presented below:

• configuration - it delivers XML Schema for the main configuration filetogether with a set of functions for parsing the file during Web Serviceruntime;

• serialization and deserialization - it utilizes advanced gSoap routines forserialization and deserialization of XML data [16];

• reconfiguration - loadable modules are supported, so various modules canbe specified in the configuration file and loaded automatically;

• transport modules - SMOA Core wraps gSoap and libpurple [19] libraries

9

for networked applications, thus HTTP, HTTPS and XMPP in SMOANotification are supported;

• authentication modules - it is easy to use various security modules forservices and clients;

• database modules - a set of functions supporting highly efficient access toexternal database over the ODBC interface.

As it was mentioned above, in addition to widely adopted HTTP and HTTPStransport protocols in Web Services, SMOA Notification supports XMPP to im-prove its interoperability and applicability. Please note that support for XMPPgives a unique opportunity for connections and asynchronous notifications withnon Web Services clients. Moreover, many XMPP servers existing in the In-ternet support highly efficient message exchange among their clients. UsingSMOA Notification together with XMPP servers as an alternative transportmedium for notifications enables users to adopt our solution to many real dis-tributed IT systems. Thanks to XMPP based communication, all notificationparties, namely Subscribers, Publishers, NotificationConsumers and SMOA No-tification itself, can be located behind firewalls with only well defined outgoingTCP/IP port open for the XMPP transport communication. Note that it is nonfunctional requirement, not defined in WS-Notification, however, security andfirewall configuration are key issues in real applications. In order to facilitatethe use of SMOA Notification in various Web Services environments, we alsoimplemented several useful libraries, tools and plugins for users and developers.There are software development kits (SDK) for C, Java and .Net which allowusers to take advantage of WS-Notification APIs as well as useful plugins forXMPP communications, for instance Pidgin-based SMOA Notification clientand GUIs [20].

5. Functional comparison of WS-Notification implementations

There is a number of available notification frameworks implementing WS-Notification standard version 1.3. As each of them has certain advantages anddisadvantages, it is difficult to select the best one taking into account both func-tional and non functional criteria. However, in this paper we try to compareselected notification systems to help users to distinguish and then select an ap-propriate solutions that could fit to their requirements. Table 2 presents theresult of our functional comparison between SMOA Notification and other pop-ular notification frameworks, namely Apache ServiceMix (version 3.3.1), IBMWebSphere (version 7.0) and Globus Toolkit (version 4.2). A set of analyzedfunctional features covers most of the WS-Notification concepts and it allowsus to highlight fundamental differences among existing standard-compliant no-tification systems. In the next subsections we describe briefly main features ofselected solutions.

10

Table 2: Comparison of notification frameworks implementing WS-Notification

SMOA Noti-fication

Apache Ser-viceMix

IBM Web-Sphere

GlobusToolkit4.x/5.0

Implementationlanguage

C Java Java C / Java

Type Brokered Brokered Brokered BaseTopic Names-paces

Yes No Yes Yes (only flat)

Dialect Full Simple Full SimpleDynamic Top-ics

Yes (TopicNamespaces)

Yes Yes Yes

Message Filters Yes Yes Yes NoPullPoints Yes Yes Yes NoCore functions SMOA Core,

XMPPJBI ServiceBus, JMS

EnterpriseService Bus,JMS

Java WS -Core, C WS -Core

5.1. Core functionsFirst, it is important to understand basic and core functions used in no-

tification frameworks. Although they are not related to the WS-Notificationstandard or specific for notification scenarios, must be concerned. Apache Ser-viceMix and WebSphere implementations use compound messaging buses thatprovide advanced mechanisms for communication, including reliability of themessages and support for a number of WS-* standards (i.e. WS-Security). Themessaging buses include also a support for HTTP/HTTPS and JMS transportsfor SOAP messages. Globus Toolkit core components, namely Java WS Coreand C WS Core, are in our opinion less complex and do not provide advancedreliability features comparing to Apache ServiceMix and WebSphere. Neverthe-less, Globus Toolkit supports many WS-* standards. SMOA Notification, inturn, is based on the SMOA Core library which is a very lightweight, but at thesame time highly efficient, solution. Additionally, SMOA Core implements WS-Addressing, WS-Security and, more importantly, it offers possibility to pluginnew transport protocols based on internal templates, e.g. used by the XMPPtransport internal plugin. As it was emphasized in the introduction section,there are unquestionable advantages of brokered type of notifications. SMOANotification belongs to a few existing notification systems that fully implementWS-BrokeredNotification. Among notification systems selected for the compar-ison in this paper, Globus Toolkit (including the latest release 5.0) does notsupport the brokered notification scenarios.

5.2. Programming languagesOne of the most important features of a notification system is its efficiency.

Naturally, a programing language linking third party communication libraries

11

impact a lot the system performance. SMOA Notification and some GlobusToolkit services were developed using the C/C++ language, while other dis-cussed in this paper notification frameworks were based on Java technologies.Obviously, there are many advantages behind the Java platform as it is aportable and easy to learn programming language. Moreover, thanks to va-riety of third party software and high level APIs for message exchange and WebServices, Java is still one of the key programming language used in Web Servicesenvironments. Nevertheless, there are also many disadvantages of using Java,especially when looking from the performance perspective. As we demonstratein the next sections, carefully selected C/C++ libraries and communicationtools that we used in SMOA Notification can provide an efficient alternative toJava based notification frameworks.

5.3. Topic FiltersWhen a number of different notification messages increases, an efficient fil-

tering of notification messages becomes a critical issue, especially in many pro-ductive distributed IT systems. According to WS-Notification the first level ofnotification filtering should be based on the mentioned concept of Topics. Web-Sphere and SMOA Notification are probably the most advanced solutions atthis point. Both systems support hierarchical TopicNamespaces as well as theFull type of TopicDialect defined in WS-Notification. Apache ServiceMix andGlobus Toolkit, in contrast, allow to use only the simplest type of the dialect- Simple. All the analyzed systems offer possibility to dynamically add newTopics (in runtime). SMOA Notification provides methods to dynamically addor remove entire Topic Namespaces.

5.4. MessageContent FiltersThe another option of notification filtering is based on the content of notifi-

cation messages. In this approach, according to WS-Notification, every messageshould be analyzed by a certain XPath expression defining key filtering words.Obviously, executing XPath queries to find key words is time consuming, buton the other hand it gives users much better flexibility required in many noti-fication scenarios where Topics are not sufficient. MessageContent Filters arenow supported in SMOA Notification, Apache ServiceMix and WebSphere.

5.5. PullPointsPullPoints enable users to use an additional synchronous way of distributing

notification messages to NotificationConsumers. Although this feature can betreated as a nice extension in sophisticated asynchronous notification scenarios,in some practical settings can be very useful. As an example we can simplyimagine a typical internal IT infrastructure protected by firewalls blocking in-coming networking connections. If NotificationConsumers are located behindsuch firewalls, one of solutions to receive notification from a notification sourceplaced in a remote external network may be pulling the notification source pe-riodically. Among selected notification frameworks, Globus Toolkit does notsupport PullPoints.

12

All the compared notification frameworks at first glance offer similar notifi-cation capabilities according to the WS-Notification standard. However, bothfunctional and non functional features discussed until now revealed some differ-ences among systems. In our opinion, taking into account not only functionalrequirements defined in WS-Notification, but also other notification capabilities,interoperability features and the software maturity, the most advanced solutionis WebSphere built on the top of IBM Enterprise Service Bus. However, pleasenote that it is a commercial product, not a publicly open software platform.On the second position, we would place both Apache ServiceMix and SMOANotification. Depending on users requirements, we believe that one of thesetwo open source solutions in many cases can easily replace the functionality ofWebSphere regarding notifications. We would like to stress the fact that evenif SMOA Notification in some aspects is less complex than Apache ServiceMix,it can perfectly fit to Web Services environments where easy to install and in-tegrate solution is required. With respect to notification scenarios taken fromWS-Notification, other existing solutions, such as Globus Toolkit or ApacheMUSE, either not support relevant notification schemas or target different mes-sage exchange scenarios. For this reason also, Globus Toolkit was excluded forfurther performance tests presented in the next sections. To conclude this partof the document, it is worth to mention here that we successfully managed tointegrate SMOA Notification with other tested systems in different notificationsettings, what simply proofs that WS-Notification can enable and then improvethe notification interoperability among different parties.

6. Performance tests

In this section, we present results of various performance tests of three noti-fication frameworks that successfully implemented three main WS-Notificationspecifications. Obviously, a reliable performance test of any distributed notifica-tion scenario must be repeatable and stable, thus some assumptions were takenduring our tests. The main assumption was related to the measurement proce-dure. In our results we always refer to the average notification delivery time tominimize the impact of random events. Please note that the distribution timeof single and/or small SOAP notification message can be disrupted by underly-ing best effort transport protocols, network protocols, services, etc. In order toperform a reliable benchmark that can be used to compare different notificationframework we created a dedicated testbed. As the notification systems we havetested are different: each system provides a set of specific functions that is notalways the same in the another systems (for example, ServiceMix doesnt providesupport for Full topic dialect) we decided to test the most common features andfocus on key scenarios. The established testbed connected two homogenous,powerful machines over the 1Gb Ethernet as it is presented in Figure 3.

Example Publisher and NotificationConsumer were deployed on the first ma-chine, whereas the second machine hosted three tested notification frameworkstogether with the another NotificationConsumer instance to allow us simplelocal tests. It is also an important assumption to compare performance tests

13

Node 18 x Intel Xeon E5345 2.33GHz

12GB RAM, GNU Linux i686 Kernel 2.6.24

8 x Intel Xeon E5345 2.33GHz12GB RAM, GNU Linux i686

Kernel 2.6.24

Ethernet 1Gb

Node 2

SMOA Notification

WebSphere Application Server

Apache ServiceMix

Notification Consumer

Notification Consumer

Publisher

Figure 3: Testbed for performance tests

to a reference point. In our case, such a reference point was created based onthe easiest notification scenarios where SOAP messages were sent directly fromPublisher to NotificationConsumer located either on the same machine or viathe network without any notification system involved. All the average valueswere calculated on 1000 repeated computing experiments, where the payload ofthe exchanged notification messages, including a Topic definition, was the con-stant value 320 bytes. In general we performed three notification tests. In thefirst two tests we measured delays during the process of forwarding notificationmessages in different notification systems, without and with active subscriptionsfor 2 Topics respectively. The main aim of the third performance test was tomeasure the subscription creation time in notification systems, as it is a com-mon action performed by notification consumers. Obtained results from theperformance tests are presented on three figures below (Figure 4, Figure 5, andFigure 6).

All three figures clearly show that in proposed setup SMOA Notification ismuch more efficient than the two other Java based notification frameworks, Web-Sphere and Apache ServiceMix. According to our tests, sending 1000 notifica-tion messages to SMOA Notification took around 0.8 second, with the overheadonly around 0.2 second comparing to the direct message exchange between Pub-lisher and NotificationConsumer. To our surprise the WebSphere notificationframework also behaved very well being able to serve 1000 notification messagesin less than 1.7 seconds, whereas Apache ServiceMix process the same amount ofnotifications in more than 4 seconds (see Figure 4). We obtained similar resultsin the second tested notification scenario, where two subscriptions for 2 Top-ics were added and we measured overall time of transporting notifications fromPublisher through the tested service to NotificationConsumer. Again, the mostefficient was SMOA Notification with the subscription overhead of only 2 per-cents, Apache ServiceMix achieved the overhead around 7 percents, and finallyWebSphere was around 11 percents slower (see Figure 5). The last presentedfigure (Figure 6) shows the average total time of creating 100 subscriptions intested systems. SMOA Notification also in this performance test outperformedits competitors, and it was around 9 times faster than WebSphere, and morethan one hundred times faster than Apache ServiceMix.

14

0.4370.643

4.288

1.683

0.796

0.0

1.0

2.0

3.0

4.0

5.0

no/fydirectlytoalocalconsumer

no/fydirectlytoaremoteconsumer

no/fyviaServiceMix no/fyviaWebSphere no/fyviaSMOANo/fica/on

Second

s

Figure 4: Average processing time of 1000 notification messages over SOAP/HTTP

7. Future Work

Although SMOA Notification is a relatively mature solution supporting ad-vanced techniques for publish/subscribe based SOAP notifications there aremany possibilities for improvements and extensions. We listed below five mainactivities that are part of the future roadmap we defined for SMOA Notification.

• Improvements to low-level communication capabilities. In addition to ba-sic reliable communication features of XMPP, we want to add internalqueuing mechanisms to be able to group several notifications within asingle notification message as well as to manage undelivered notificationsand respond to peaks generated by Publishers. Such mechanisms are notexplicitly mentioned in WS-Notification, but have been implemented inWebSphere improving its reliability. There are also other optimizationtechniques possible, e.g. proposed by Quiroz and Parashar [14], or othertransport level protocols that can be added, e.g. SMTP.

• Additional high-level interfaces. Currently SMOA Notification supportsWeb Services interfaces according to WS-Notification. However, we be-lieve that SMOA Notification core functions are robust and flexible enoughto be used for extended high-level Web Services interfaces, in particular tosupport the WS-Eventing standard. A very interesting solution linking to-gether notification concepts defined by WS-Notification and WS-Eventingwas presented in [13].

15

4.586

1.876

0.885

0.0

1.0

2.0

3.0

4.0

5.0

no.fyviaServiceMix no.fyviaWebSphere no.fyviaSMOANo.fica.on

Second

s

Figure 5: Average processing time of 1000 notification messages over SOAP/HTTP with thesubscription overhead

• Fault tolerance improvements. The automatic redistribution of notifica-tion objects, such as Topics or Subscriptions, among many notificationservices located on distributed machines in a network is a very interestingtopic as it was discussed in [14]. In our opinion it is possible to build such adistributed notification systems still taking advantage of WS-Notification.We plan to setup various testing environments in which based on simpleextensions to SMOA Notification many notification services will be able toexchange and duplicate internal notification structures and thus improvethe fault tolerance of the whole system. Moreover, we also plan to performmore scalability tests aiming at highly distributed IT systems.

• New administrative and client tools and GUIs. Finally, in case more userswill use and contribute to SMOA Notification, we plan to distribute oursystem together with various new both administrative and client tools tosimplify the integration and management of notification delivery.

8. Conclusions

Today, building new or integrating existing SOA-based systems using ap-propriate notification mechanisms is still a challenging task. In this paper we

16

22.147

1.510

0.168

0.1

1.0

10.0

subscribeinServiceMix subscribeinWebSphere subscribeinSMOANo>fica>on

Second

s

Figure 6: Average subscription creation time of 100 subscriptions

shared our experiences with WS-Notification industry standards and their ref-erence implementations which can be used to reduce this effort. Taking intoaccount a large number of distributed Service-Oriented systems using notifica-tion capabilities for service composition and integration we decided to compareour approach to the most popular WS-Notification systems. Additionally, basedon both functional and non-functional comparisons as well as many experimen-tal tests, we demonstrated that our solution can easily compete with well knownnotification frameworks, such as WebSphere or Apache ServiceMix. We believethat in some applications, especially where the efficiency plays a key role, oursystem can be easily use to improve or even reduce various communicationbottlenecks. SMOA Notification offers an attractive alternative for many SOA-based system developers dealing with the service integration and composition.As a proof of concept, it has been successfully used together with QosCosGridmiddleware services to create Pan-European computing testbeds for large-scalecomplex system simulations. Finally, SMOA Notification as an easy-to-use noti-fication system comparing to sophisticated notification frameworks fulfills manyrequirements for event-based scenarios in emerging SOA environments, in par-ticular:

• well defined and standard message exchange scenarios, by implementing

17

all OASIS WS-Notification industry standards and schemas;

• dynamic reconfiguration of the environment, by supporting independentnotification entities, such as Publishers, Subscribers, NotificationCon-sumers that can be dynamically added to the system;

• topic-based notifications and content flittering, by allowing users to selectand use appropriate flittering mechanisms, including support for XPathqueries;

• efficiency, outperforming existing notification frameworks in typical mes-sage delivery and notification actions;

• extensibility and configurability, by offering a modular architecture wherenew plugins (e.g XMPP/jabber based notifications), and extensions canbe easily added at both low and high levels.

9. Acknowledgments

The SMOA Notification system has been developed and used under the twoEU funded FP6 projects, namely BREIN [21] and QosCosGrid [22]. In the firstproject, powerful notification capabilities of SMOA Notification were demon-strated in asynchronous and dynamic message exchange in a distributed ITsystem controlling some events generated by an example airport managementsystem, such as flight delays, SLA violations, etc. In the QosCosGrid we adoptedSMOA Notification together with other Web Services using the SMOA frame-work, in particular SMOA Computing, to build highly efficient grid middlewarefor cross-cluster parallel application execution and monitoring. Promising re-sults encouraged users to use SMOA Notification in many other IT systems andenvironments, and the system itself is a subject of software productive qualitytests under the national grid infrastructure initiative called PL-GRID [23].

References

[1] S. Pallickara, G. C. Fox, An Analysis of Notification Related Specificationsfor Web/Grid applications, Tech. Rep., Indiana Univeristy, Bloomington,IN, URL http://grids.ucs.indiana.edu/ptliupages/publications/WS-NotifyEventComparison.pdf, 2004.

[2] D. Box, C. Critchley, F. Curbera, D. Ferguson, S. Graham, D. Hull,G. Kakivaya, A. Lewis, B. Lovering, P. Niblett, D. Orchard,S. Samdarshi, J. Schlimmer, I. Sedukhin, J. Shewchuk, S. Weer-awarana, D. Wortendyke, Web Services Eventing (WS-Eventing), URLhttp://www.w3.org/Submission/Ws-Eventing, 2006.

[3] OASIS WSN Technical Committee, Web Services Notification 1.3 OASISStandard, URL http://www.oasis-open.org/committees/wsn, 2006.

18

[4] B. Kowalewski, M. Bubak, B. Balis, An Event-Based Approach to Re-ducing Coupling in Large-Scale Applications., in: M. Bubak, G. D.van Albada, J. Dongarra, P. M. A. Sloot (Eds.), ICCS (3), vol.5103 of Lecture Notes in Computer Science, Springer, ISBN 978-3-540-69388-8, 358–367, URL http://dblp.uni-trier.de/db/conf/iccS/iccS2008-3.html#KowalewskiBB08, 2008.

[5] Object Management Group, Notification Service Specification, URLhttp://www.omg.org/spec/NOT/, undated.

[6] M. Hapner, R. Burridge, R. Sharma, J. Fialli, K. Stout, Java MessageService, URL http://java.sun.com/products/jms/, undated.

[7] Patrick, P. A. Felber, R. Guerraoui, A. M. Kermarrec, The Many Faces ofPublish/Subscribe, Tech. Rep. DSC ID:200104, EPFL, 2001.

[8] Y. Huang, D. Gannon, A Comparative Study of Web Services-based EventNotification Specifications, in: ICPPW ’06: Proceedings of the 2006 In-ternational Conference Workshops on Parallel Processing, IEEE Com-puter Society, Washington, DC, USA, ISBN 0-7695-2637-3, 7–14, doi:http://dx.doi.org/10.1109/ICPPW.2006.5, 2006.

[9] I. Foster, Globus Toolkit Version 4: Software for Service-OrientedSystems, Journal of Computer Science and Technology 21 (4)(2006) 513–520, ISSN 1000-9000, doi:10.1007/s11390-006-0513-y, URLhttp://dx.doi.org/10.1007/s11390-006-0513-y.

[10] The Apache Software Fundation, Apache Muse, URLhttp://ws.apache.org/muse/, undated.

[11] The Apache Software Fundation, Apache ServiceMix, URLhttp://servicemix.apache.org, undated.

[12] A. Agopyan, H. Huebler, T. Puah, T. Schulze, D. S. Vilageliu, M. Keen,WebSphere Application Server V7.0: Concepts, Planning, and Design, URLhttp://www.redbooks.ibm.com/redbooks/pdfs/sg247708.pdf, 2009.

[13] Y. Huang, S. A., H. C., G. D., WS-Messenger: a Web services-based mes-saging system for service-oriented grid computing, in: Proc. Sixth IEEEInternational Symposium on Cluster Computing and the Grid CCGRID06, vol. 1, 8pp., doi:10.1109/CCGRID.2006.109, 2006.

[14] A. Quiroz, M. Parashar, Design and Implementation of a DistributedContent-based Notification Broker for WS-Notification, in: Proc. thIEEE/ACM International Conference on Grid Computing, 207–214, doi:10.1109/ICGRID.2006.311017, 2006.

[15] M. R. Head, M. Govindaraju, A. Slominski, P. Liu, N. Abu-Ghazaleh,R. van Engelen, K. Chiu, M. J. Lewis, A Benchmark Suite for SOAP-based Communication in Grid Web Services, in: Proc. ACM/IEEE SC2005 Conference Supercomputing, 19, doi:10.1109/SC.2005.2, 2005.

19

[16] R. A. Van Engelen, K. A. Gallivan, The gSOAP Toolkit for Web Ser-vices and Peer-to-Peer Computing Networks, in: Proc. 2nd IEEE/ACMInternational Symposium on Cluster Computing and the Grid, 128, doi:10.1109/CCGRID.2002.1017120, 2002.

[17] P. Saint-Andre, K. Smith, R. Tronon, XMPP: The Definitive Guide Build-ing Real-Time Applications with Jabber Technologies, O’Reilly Media, Inc.,ISBN 059652126X, 9780596521264, 2009.

[18] K. Kurowski, M. Mamonski, B. Bosak, SMOA project, URLhttp://sourceforge.net/projects/smoa-project/, undated.

[19] Pidgin Community, What is llibpurple?, URLhttp://developer.pidgin.im/wiki/WhatIsLibpurple, undated.

[20] Pidgin Community, Pidgin, URL http://www.pidgin.im/, undated.

[21] BREIN Consortium, BREIN - Business objective driven reliable and intel-ligent grids for real business, URL http://www.eu-brein.com/, undated.

[22] K. Kurowski, W. Back, W. Dubitzky, L. Gulyas, G. Kampis, M. Mamonski,G. Szemes, M. Swain, Complex System Simulations with QosCosGrid, in:ICCS ’09: Proceedings of the 9th International Conference on Computa-tional Science, Springer-Verlag, Berlin, Heidelberg, ISBN 978-3-642-01969-2, 387–396, doi:http://dx.doi.org/10.1007/978-3-642-01970-8 38, 2009.

[23] PL-Grid Consortium, PL-Grid, URL http://www.plgrid.pl, undated.

20