applying patterns to develop extensible orb middleware
TRANSCRIPT
Applying Patterns to Develop Extensible ORB Middleware
Douglas C. Schmidt and Chris Cleelandfschmidt,[email protected]
Department of Computer ScienceWashington University
St. Louis, MO 63130, USA(314) 935-7538�
This article has been submitted to IEEE CommunicationsMagazine, special issue on design patterns.
Abstract
Distributed object computing forms the basis for next-generation application middleware. At the heart of distributedobject computing are Object Request Brokers (ORBs), whichautomate many tedious and error-prone distributed program-ming tasks. This article presents a case study of key designpatterns needed to develop ORBs that can be dynamically con-figured and evolved for specific application requirements andsystem characteristics.
1 Introduction
Four trends are shaping the future of commercial software de-velopment. First, the software industry is moving away fromprogrammingapplications from scratch tointegratingapplica-tions using reusable components [1] such as ActiveX and JavaBeans. Second, there is great demand fordistribution tech-nologysuch as remote method invocation or message-orientedmiddleware that simplifies application collaboration withinand across enterprises [2]. Third, there are increasing efforts todefine standard software infrastructures such as CORBA thatallow applications to interwork seemlessly throughouthetero-geneousenvironments [3]. Finally, next-generation distributedapplications such as video-on-demand, teleconferencing, andavionics require Quality of Service (QoS) guarantees for la-tency, bandwidth, and reliability [4].
A key software technology supporting these trends isdis-tributed object computing (DOC) middleware. DOC middle-ware facilitates the collaboration of local and remote applica-tion components in heterogeneous distributed environments.
�This research is supported in part by grants from Boeing, NSF grantNCR-9628218, Siemens, and Sprint.
The goal of DOC middleware is to eliminate many tedious,error-prone, and non-portable aspects of developing and main-taining distributed applications. In particular, DOC middle-ware automates common network programming tasks such asobject location, object activation, parameter marshaling, faultrecovery, and security. At the heart of DOC middleware areObject Request Brokers(ORBs), such as CORBA [5], DCOM[6], and Java RMI [7].
This article describes howdesign patternscan be used todevelop dynamically configurable ORB middleware that canbe extended and maintained more easily than statically con-figured middleware. A pattern represents a recurring solutionto a software development problem within a particular context[8]. Patterns help to alleviate the continual re-discovery andre-invention of software concepts and components by docu-menting proven solutions to standard software developmentproblems [9]. For instance, patterns are useful for document-ing the structure and participants in common communicationsoftware micro-architectures like Reactors [10], Active Ob-jects [11], and Brokers [12]. These patterns are generaliza-tions of object-structures that have been used successfully tobuild flexible and efficient event-driven and concurrent com-munication software such as ORBs.
To focus the discussion, this article presents a case studythat illustrates how we have applied patterns to developTheACE ORB(TAO) [13]. TAO is a freely available, highly exten-sible ORB targeted for applications with real-time quality ofservice (QoS) requirements. These applications include avion-ics mission computers [14], telecommunication switch man-agement systems [10], and electronic medical imaging sys-tems [15]. A novel aspect of TAO is its extensible design,which can be customized dynamically to meet specific appli-cation QoS requirements and network/endsystem characteris-tics.
1
2 Overview of CORBA and TAO
2.1 Overview of the CORBA ORB ReferenceModel
CORBA Object Request Brokers (ORBs) [3] allow clients toinvoke operations on distributed objects without concern forobject location, programming language, OS platform, commu-nication protocols and interconnects, and hardware. Figure 1illustrates the components in the CORBA reference model,which collaborate to provide the portability, interoperability,and transparency mentioned above.
Figure 1 illustrates the components in the CORBA refer-ence model, all of which collaborate to provide the portability,interoperability, and transparency outlined above.
DIIDII ORBORBINTERFACEINTERFACE
ORBORBCORECORE
operation()operation()
OBJECTOBJECT
ADAPTERADAPTER
IDLIDLSKELETONSKELETON
DSIDSI
in argsin args
out args + return valueout args + return value
CLIENTCLIENT
GIOPGIOP//IIOPIIOP
SERVANTSERVANT
STANDARD INTERFACESTANDARD INTERFACE STANDARD LANGUAGESTANDARD LANGUAGE
MAPPINGMAPPING
ORB-ORB-SPECIFIC INTERFACESPECIFIC INTERFACE STANDARD PROTOCOLSTANDARD PROTOCOL
IDLIDLSTUBSSTUBS
Figure 1: Components in the CORBA Reference Model
The client-side stubs marshal client operations into requeststhat are transmitted to servants via the standard Internet Inter-ORB Protocol (IIOP) implemented in the ORB Core. Theserver-side ORB Core receives these requests and passes themto the Object Adapter. The Object Adapter demultiplexes therequests to skeletons, which demarshal the requests and dis-patch the appropriate application-level servant operation.
2.2 Overview of TAO
TAO is an ORB endsystem that contains the network interface,operating system, communication protocol, and CORBA mid-dleware components and features shown in Figure 2. TAO isbased on the standard OMG CORBA reference model, withthe following enhancements designed to overcome the short-comings of conventional ORBs for high-performanceand real-time applications:
Real-time IDL Stubs and Skeletons: In addition to mar-shaling and demarshaling of operation parameters, TAO’s
NETWORKNETWORK
ORBORB QQOOSSINTERFACEINTERFACE
REALREAL--TIMETIMEORBORB CORECORE
operation()operation()
RIDLRIDLSTUBSSTUBS
REALREAL--TIMETIME
OBJECTOBJECT
ADAPTERADAPTER
RIDLRIDLSKELETONSKELETON
in argsin args
out args + return valueout args + return value
CLIENTCLIENT
OS KERNELOS KERNEL
HIGHHIGH--SPEEDSPEED
NETWORK ADAPTERSNETWORK ADAPTERS
REALREAL--TIME ITIME I//OOSUBSYSTEMSUBSYSTEM
RIOPRIOP
SERVANTSERVANT
OS KERNELOS KERNEL
HIGHHIGH--SPEEDSPEED
NETWORK ADAPTERSNETWORK ADAPTERS
REALREAL--TIME ITIME I//OOSUBSYSTEMSUBSYSTEM
Figure 2: Components in the TAO Real-time ORB
Real-time IDL (RIDL) stubs and skeletons are designed to en-sure that application timing requirements are specified and en-forced end-to-end [16].
Real-time Object Adapter and ORB Core: In addition toassociating servants with the ORB and demultiplexing incom-ing requests to servants, TAO’s Object Adapter (OA) imple-mentation dispatches servant operations in accordance withvarious real-time scheduling strategies such as Rate Mono-tonic and Maximal Urgency First [13].
ORB QoS Interface: TAO’s QoS interface is designedto map real-time processing requirements to ORB endsys-tem/network resources. Common real-time processing re-quirements include end-to-end latency bounds and periodicscheduling deadlines. Common ORB endsystem/network re-sources include CPU, memory, network connections and stor-age devices.
Real-time I/O subsystem: TAO’s real-time I/O subsystemperforms admission control and assigns priorities to real-timeI/O threads so that the schedulability of application compo-nents and ORB endsystem resources can be guaranteed.
High-speed network adapters: TAO’s I/O subsystem con-tains a “daisy-chained” interconnect comprising a number ofATM Port Interconnect Controller (APIC) chips [17]. APICis designed to sustain an aggregate bi-directional data rate of2.4 Gbps. However, TAO also runs on conventional real-timeinterconnects such as VME backplanes and multi-processorshared memory environments.
2
3 Using Patterns to Build an ExtensibleORB Middleware
This section uses TAO as a case study to illustrate how patternscan help application developers and ORB developers build,maintain, and extend communication software by reducing thecoupling between components. The patterns described in thissection are not limited to ORBs or communication middle-ware, however. They have been applied in many other commu-nication application domains, including telecom call process-ing and switching, avionics flight control systems, multimediateleconferencing, and distributed interactive simulations.
Figure 3 illustrates the patterns used to develop an extensi-ble ORB architecture for TAO. It is beyond the scope of this
ACCEPTORCONNECTOR
ABSTRACT
FACTORY
SERVANTCLIENT
OS KERNELOS KERNEL
ACTIVE
OBJECT
THREAD-SPECIFIC
STORAGE
SERVICE
CONFIGURATOR
STRATEGY
REACTORREACTOR
WRAPPER FACADESWRAPPER FACADES
Figure 3: Relationships Among Patterns Used in TAO
section to describe each pattern in detail or to discuss all thepatterns used within TAO. Instead, our goal is to focus on thekey patterns and show how they can improve the extensibil-ity, maintainability, and performance of complex ORB mid-dleware. The references point to additional material on eachpattern.
In the following discussion, we outline the forces that un-derlie the main design challenges related to developing ex-tensible and maintainable ORBs. We also explain how theabsence of these patterns leaves these forces unresolved. Inaddition, we describe which patterns resolve these forces andillustrate how TAO implements these patterns to create an ex-tensible and maintainable real-time ORB.
3.1 Encapsulate Low-level System Mechanismswith the Wrapper Facade Pattern
Context: One role of an ORB is to shield application-specific clients and servants from the details of low-level sys-tems programming. Thus, ORB developers, rather than appli-cation developers, are responsible for tedious, low-level taskslike demultiplexing events, sending and receiving requestsfrom the network, and spawning threads to execute requestsconcurrently.
Problem: Developing an ORB is difficult. It is even moredifficult if developers must wrestle with low-level system APIswritten in languages like C, which often causes the followingproblems:
� ORB developers must have intimate knowledge ofmany OS platforms: Implementing an ORB using system-level APIs forces developers to deal with the non-portable,tedious, and error-prone OS idiosyncrasies such as using un-typed socket handles to identify connection endpoints. More-over, these APIs are not portable across OS platforms. Forexample, Win32 lacks Pthreads and has subtly different se-mantics for sockets andselect .
� Increased maintenance effort: One way to build anORB is to handle portability variations via explicit conditionalcompilation directives in ORB source code. Using condi-tional compilation to address platform-specific variationsatall points of useincreases the complexity of the source code.It is hard to maintain and extend such ORBs, however, sinceplatform-specific details are scattered throughout the imple-mentation files.
� Inconsistent programming paradigms: System mech-anisms are accessed through C-style function calls, whichcause an “impedance mismatch” with the OO programmingstyle supported by C++, the language used to implement TAO.
How can we avoid accessing low-level system mechanismswhen implementing an ORB?
Solution ! the Wrapper Facade pattern: An effectiveway to avoid accessing system mechanisms directly is to usethe Wrapper Facade pattern. This pattern is a variant of theFacade pattern [8]. The intent of the Facade pattern is to sim-plify the interface for a subsystem. The intent of the WrapperFacade pattern is more specific: it provides type-safe, modu-lar, and portable class interfaces that encapsulate lower-level,stand-alone system mechanisms such as sockets,select ,and Pthreads. In general, the Wrapper Facade pattern shouldbe applied when existing system-level APIs are non-portableand non-type-safe.
3
Using the Wrapper Facade pattern in TAO: TAO accessesall system mechanisms via the wrapper facades provided byACE [18]. ACE is an OO framework that implements coreconcurrency and distribution patterns for communication soft-ware. It provides reusable C++ wrapper facades and frame-work components that are targeted to developers of high-performance, real-time applications and services across a widerange of OS platforms, including Win32, most versions ofUNIX, and real-time operating systems (like VxWorks, Cho-rus, and LynxOS).
Figure 4 illustrates how the ACE C++ wrapper facadesimprove TAO’s robustness and portability by encapsulatingand enhancing native OS concurrency, communication, mem-ory management, event demultiplexing, and dynamic linkingmechanisms with type-safe OO interfaces. The OO encap-
TAO's ORB CoreTAO's ORB Core
spawn()spawn()
acquire()acquire()
open(),open(),close(),close(),
recv(), send()recv(), send()
dlopen()dlopen()
dlsym()dlsym()
COMMUNICATIONCOMMUNICATION
SUBSYSTEMSUBSYSTEM
VIRTUALVIRTUAL
MEMORY MEMORY
SUBSYSTEMSUBSYSTEM
GENERALGENERAL
POSIXPOSIX && WWININ3232
SERVICESSERVICES
PROCESSPROCESS//THREADTHREAD
SUBSYSTEMSUBSYSTEM
THREADTHREAD
WRAPPERSWRAPPERS
DYNAMIC
LINKING
SELECT/IO COMP
SOCKETS/TLI
ACEWRAPPER
FACADES
handle_events()
Figure 4: TAO’s Wrapper Facade Encapsulation
sulation provided by ACE alleviates the need for TAO to ac-cess the weakly-typed system mechanisms directly. Therefore,C++ compilers can detect type system violations at compile-time rather than at run-time. ACE wrapper facades use C++features to eliminate performance penalties that would other-wise be incurred from its additional type-safety and layer ofabstraction. For instance, inlining is used to avoid the over-head of calling small method calls. Likewise, static methodsare used to avoid the overhead of passing athis pointer toeach invocation.
Although the ACE wrapper facades solve a common devel-opment problem, they are just the first step towards developingan extensible and maintainable ORB. The remaining patternsdescribed in this section build on the encapsulation providedby the ACE wrapper facades to address more challenging ORBdesign issues.
3.2 Demultiplexing ORB Core Events using theReactor Pattern
Context: An ORB Core is responsible for demultiplexingI/O events from multiple clients and dispatching their asso-ciated event handlers. For instance, a server-side ORB Core
listens for new client connections and reads/writes GIOP re-quests/responses from/to connected clients. To ensure re-sponsiveness to multiple clients, an ORB Core uses OSevent demultiplexing mechanisms to wait forCONNECTION,READ, andWRITE events to occur onmultiplesocket handles.Common event demultiplexing mechanisms includeselect ,WaitForMultipleObjects , I/O completion ports, andthreads.
Problem: One way to develop an ORB Core is to hard-codeit to use one event demultiplexing mechanism. Relying onjust one mechanism is undesirable, however, since no singlescheme is efficient on all platforms or for all application re-quirements. For instance, asynchronous I/O completion portsare very efficient on Windows NT [19], whereas synchronousthreads are the most efficient demultiplexing mechanism onSolaris [20].
Another way to develop an ORB Core is to tightly couple itsevent demultiplexing code with the code that performs GIOPprotocol processing. In this case, the demultiplexing code can-not be reused as a blackbox component by similar communi-cation middleware applications such as HTTP servers [19] orvideo-on-demand applications. Moreover, if new ORB strate-gies for threading or Object Adapter request scheduling algo-rithms are introduced, substantial portions of the ORB Coremust be re-written.
How then can an ORB implementation render itself inde-pendent of a specific event demultiplexing mechanism and de-couple its demultiplexing code from its handling code?
Solution ! the Reactor pattern: An effective way to re-duce coupling and increase the extensibility of an ORB Coreis to apply theReactor pattern[10]. This pattern supportssynchronous demultiplexing and dispatching of multipleeventhandlers, which are triggered by events that can arrive concur-rently from multiple sources. The Reactor pattern simplifiesevent-driven applications by integrating the demultiplexing ofevents and the dispatching of their corresponding event han-dlers. In general, the Reactor pattern should be applied whenan application like an ORB Core must handle events from mul-tiple clients concurrently, without commiting itself to a singlelow-level mechanism likeselect .
It is important to note that applying the Wrapper Facade pat-tern is not sufficient to resolve the problems outlined above. Awrapper facade forselect may improve ORB Core porta-bility somewhat. However, this pattern does not resolve theneed to completely decouple the low-level event demultiplex-ing logic from the higher-level client request processing logicin an ORB Core.
Using the Reactor pattern in TAO: TAO uses the Re-actor pattern to drive the main event loop within its ORBCore, as shown in Figure 5. A TAO server (1) initi-
4
ORB COREORB CORE
1: RUN EVENT LOOP
OBJECTOBJECT
ADAPTERADAPTER
APPLICATIONAPPLICATION
3: handle_input()
4: DISPATCH
SERVANTSERVANT
5: UPCALL
ACTIVE OBJECT MAPACTIVE OBJECT MAP
2: select()
ReactorReactor
ConnectionConnectionHandlerHandler
ConnectionConnectionHandlerHandler
ConnectionConnectionHandlerHandler
Figure 5: Using the Reactor Pattern in TAO’s Event Loop
ates an event loop in the ORB Core’sReactor , whereit (2) remains blocked onselect until an I/O event oc-curs. When a GIOP request event occurs, theReactordemultiplexes the request to the appropriate event handler,which is the GIOPConnection Handler that is associ-ated with each connected socket. TheReactor (3) then callsConnection Handler::handle input , which (4) dis-patches the request to TAO’s Object Adapter. The ObjectAdapter demultiplexes the request to the appropriate upcallmethod on the servant and (5) dispatches the upcall.
The Reactor pattern enhances the extensibility of TAO bydecoupling the event handling portions of its ORB Core fromthe underlying OS event demultiplexing mechanisms. Forexample, theWaitForMultipleObjects event demulti-plexing system call is used on Windows NT, whereasselectis used on UNIX platforms. Moreover, the Reactor patternsimplifies the configuration of new event handlers. For in-stance, adding a newSecure Connection Handler thatperforms encryption/decryption of all traffic does not affectthe Reactor’s implementation.
3.3 Managing Connections in an ORB UsingAcceptor-Connector Pattern
Context: Connection management is another key respon-sibility of an ORB Core. For instance, an ORB Corethat implements the IIOP protocol must establish TCP con-nections and initialize the protocol handlers for each IIOPserver endpoint . By localizing connection managementlogic in the ORB Core, application-specific servants are ableto focus solely on processing client requests.
An ORB Core is notlimited to running over IIOP and TCPtransports, however. For instance, while TCP can transfer
GIOP requests reliably, its flow control and congestion controlalgorithms may preclude its use as a real-time protocol [13].Likewise, it may be more efficient to use a shared memorytransport mechanism when clients and servants are co-locatedon the same endsystem. Thus, an ideal ORB Core must beflexible in its support of multiple transport mechanisms.
Problem: The CORBA architecture explicitly decouples theconnection management tasks performed by an ORB Corefrom the request processing performed by an application ser-vant. However, one way to implement an ORB’sinternalconnection management activities is to use low-level networkAPIs like sockets. Likewise, the connection establishmentprotocol can be tightly coupled with the communication pro-tocol.
This design approach yields the following drawbacks, how-ever:
1. Too inflexible: If an ORB’s connection managementdata structures and algorithms are too closely intertwined, sub-stantial effort is required to modify the ORB Core. For in-stance, tight coupling the ORB to use the socket API makesit hard to change the underlying transport mechanism,e.g., touse shared memory rather than sockets. Therefore, it is intru-sive and time consuming to port a tightly coupled ORB Coreto new networks, such as ATM, or different network program-ming APIs, such as TLI or Win32 Named Pipes.
2. Too inefficient: Many internal ORB strategies can beoptimized by allowing both ORB developers and applicationdevelopers to select appropriate implementations late in thedesign cycle,e.g., after extensive performance profiling. Forexample, a multi-threaded, real-time ORB client may needto store connection endpoints in thread-specific storage to re-duce lock contention and overhead. Similarly, the concurrencystrategy for a CORBA server might require that each connec-tion run in its own thread to eliminate per-request locking over-head. However, if connection management mechanism arehard-coded and tightly bound with other internal ORB strate-gies it is hard to accommodate efficient new strategies.
How then can an ORB Core’s connection management com-ponents support multiple transports and allow connection-related behaviors to be (re)configured flexibly late in the de-velopment cycle?
Solution! the Acceptor-Connector pattern: An effectiveway to increase the flexibility of ORB Core connection man-agement and initialization is to apply theAcceptor-Connectorpattern[21]. This pattern decouples connection initializationfrom the processing performed once a connection endpoint isinitialized. TheAcceptor component in the pattern is re-sponsible forpassiveinitialization, i.e., the server-side of theORB Core. Conversely, theConnector component in the
5
pattern is responsible foractive initialization, i.e., the client-side of the ORB Core. In general, the Acceptor-Connector pat-tern should be applied when client/server middleware must al-low flexible configuration of network programming APIs andmust maintain proper separation of initialization roles.
Using the Acceptor-Connector pattern in TAO: TAOuses the Acceptor-Connector pattern in conjunction withthe Reactor pattern to handle setup of connections forGIOP/IIOP communication. Within TAO’s client-sideORB Core, aConnector initiates connections to serversin response to a method invocation or explicit bind-ing to a remote object. Within TAO’s server-sideORB Core, one or moreAcceptor s creates a GIOPConnection Handler to service each new client connec-tion. Acceptor s andConnection Handler s both derivefrom Event Handler , which enable them to be dispatchedautomatically by aReactor .
TAO’s Acceptors andConnectors can be configuredwith any transport mechanisms, such as sockets or TLI, pro-vided by the ACE wrapper facades. In addition, TAO’sAcceptor and Connector can be imbued with customstrategies to systematically select an appropriate concurrencymechanism, as described in Section 3.4.
Figure 6 illustrates the use of Acceptor-Connector strate-gies in TAO’s ORB Core. When a client (1) invokes a
GIOPGIOP
HandlerHandler
Strategy StrategyConnectorConnector
Cached Cached
ConnectConnect
StrategyStrategy
ConnectionConnectionHandlerHandler
ReactorReactor
ORBORB CORECORE
SERVERSERVERCLIENTCLIENT
5:5: REQUEST REQUEST//RESPONSERESPONSE
2: connect()2: connect() 3: accept()3: accept()
0..N
6: DISPATCH()
4: CREATE & ACTIVATE
1: operation()
ConnectionHandler
ConnectionHandler
ConnectionHandler
Concurrency
Strategy
StrategyAcceptor
Figure 6: Using the Acceptor-Connector Pattern in TAO’sConnection Management
remote operation, it makes aconnect call through theStrategy Connector . The Strategy Connector(2) consults its connection strategy to obtain a connection. Inthis example the client uses a “caching connection strategy”that recycles connections to the server. Thus, it only createsnew connections when all existing connections are already in
use. This strategy minimizes connection setup time, therebyreducing end-to-end request latency.
In the server-side ORB Core, theReactor notifiesTAO’s Strategy Acceptor to (3) accept newly con-nected clients and createConnection Handlers . TheStrategy Acceptor delegates the choice of concurrencymechanism to one of TAO’sconcurrencystrategies (e.g.,reactive, thread-per-request, thread-per-connection, thread-per-priority, etc.) described in Section 3.4. Once aConnection Handler is activated (4) within the ORBCore, it performs the requisite GIOP protocol processing (5)on a connection and ultimately dispatches (6) the request tothe appropriate servant via TAO’s Object Adapter.
3.4 Simplifying ORB Concurrency using theActive Object Pattern
Context: Once the Object Adapter has dispatched a clientrequest to the appropriate servant, the servant executes the re-quest. Execution may occur in the same thread of control asthe Connection Handler that received it. Conversely,execution may occur in a different thread, concurrent withother request executions. The CORBA specification does notaddress the issue of concurrency within an ORB or a servant,leaving the decision to ORB developers and end-users.
It is important to develop ORBs that manage concur-rent processing efficiently. Concurrency allows long-runningoperations to execute simultaneously without impeding theprogress of other operations. Likewise, preemptive multi-threading is crucial to minimize the dispatch latency of real-time systems [14].
Problem: In many ORBs, the concurrency architecture isprogrammed directly using the OS platform’s multi-threadingAPI, such as the POSIX Pthreads API [22]. However, thereare several drawbacks to this approach:
� Non-portable: Threading APIs tend to be veryplatform-specific. Even industry standards such as POSIXPthreads are not available on many widely-used OS platforms,including Win32, VxWorks, and pSoS. Not only is there no di-rect mapping between APIs, but there is no clear mapping offunctionality. For instance, POSIX Pthreads supports deferredthread cancellation, whereas Win32 threads do not. Moreover,although Win32 has a thread termination API, but the docu-mentation strongly recommendsnot using it since it does notrelease thread resources on exit. Moreover, Pthreads itself isnon-portable since many UNIX vendors implement differentdrafts of the standard.
� Hard to program correctly: Programming a multi-threaded ORB is hard since application and ORB developersmust ensure that access to shared data is serialized properly in
6
the ORB and in the servants. In addition, the techniques re-quired to robustly terminate servants that execute concurrentlyin multiple threads are complicated, non-portable, and non-intuitive.
� Non-extensible: The choice of an ORB concurrencystrategy depends largely on external factors like applicationrequirements and network/endsystem characteristics. For in-stance, Reactive single-threading [10] is an appropriate strat-egy for short duration, compute-bound requests on a uni-processor. If these external factors change, however, it is im-portant that an ORB can be extended to handle alternative con-currency strategies such as thread-per-request, thread pool, orthread-per-priority.
When ORBs are developed using low-level threading APIs,however, they are hard to extend it with new concurrencystrategieswithoutaffecting other ORB components. How thencan an ORB support a simple, extensible, and portable concur-rency mechanism?
Solution! the Active Object pattern: An effective way toincrease the portability, correctness, and extensibility of ORBconcurrency strategies is to apply theActive Object pattern[11]. This pattern provides a higher-level concurrency archi-tecture that decouples the thread that initially receives and pro-cesses a client request from the thread that ultimately executesthe request.
While Wrapper Facadesprovide the basis for portability,they are simply thin veneers over the low-level system mech-anisms. Moreover, a facade’s behavior may still vary acrossplatforms. Therefore, the Active Object pattern defines ahigher-level concurrency abstraction that shields TAO fromthe complexity of low-level thread facades. By raising thelevel of abstraction for ORB developers, the Active Object pat-tern makes it easier to define more portable, flexible, and easyto program ORB concurrency strategies.
In general, the Active Object pattern should be used whenan application can be simplified by centralizing the pointwhere concurrency decisions are made. This pattern givesdevelopers the flexibilityx to insert decision points betweeneach request’s initial reception and its ultimate execution. Forinstance, developers could decide whether or not to spawn athread-per-connection or a thread-per-request.
Using the Active Object pattern in TAO: TAO usesthe Active Object pattern to transparently allow a GIOPConnection Handler to execute requests eitherreac-tivelyby borrowing the Reactor’s thread of control oractivelyby running in its own thread of control. The sequence of stepsis shown in Figure 7.
The processing shown in Figure 7 is triggered when (1) aReactor notifies theConnection Handler that an I/Oevent is pending. Based on the currently configured strat-egy, e.g., reactive, thread-per-connection, thread-per-request,
Concurrency ConcurrencyStrategyStrategy
ReactorReactor
ORB COREORB CORE
2:2: ACTIVE OR PASSIVE ACTIVE OR PASSIVE??
ConnectionConnectionHandlerHandler
2a: Task::activate()2a: Task::activate()
1: handle_input()1: handle_input()
3:3: SERVICE REQUEST SERVICE REQUEST
Figure 7: Using the Active Object Pattern to Structure TAO’sConcurrency Strategies
etc., the handler (2) determines if it should be active orpassive and acts accordingly. This flexibility is achievedby inheriting TAO’s ORB Core connection handling classesfrom an ACE base class calledTask . To process a requestconcurrently, therefore, the handler simply (2a) invokes theTask::activate method. This method spawns a newthread and invokes a standard hook method. Whether activeor passive, the handler will ultimately (3) process the request.
3.5 Reducing Lock Contention and Priority In-versions with the Thread-Specific StoragePattern
Context: The Active Object pattern allows applications andcomponents in the ORB to operate using a variety of concur-rency strategies, rather than one enforced by the ORB itself.The primary drawback to concurrency, however, is the need toserialize access to shared resources, such as operatorsnew anddelete , pointers created by theCORBA::ORBinit ORBinitialization factory, or theAcceptor andConnector de-scribed in Section 3.3. A common way to achieve serializa-tion is to use mutual-exclusion locks on each resource sharedby multiple threads. However, acquiring and releasing theselocks can be expensive, often negating any potential perfor-mance benefits of concurrency.
Problem: In theory, multi-threading an ORB can improveperformance by executing multiple instruction streams simul-taneously. In addition, multi-threading can simplify inter-nal ORB design by allowing each thread to execute syn-chronously rather than reactively or asynchronously. In prac-tice, multi-threaded ORBs often perform no better, or evenworse, than single-threaded ORBs due to (1) the cost of acquir-ing/releasing locks and (2) priority inversions that arise whenhigh- and low-priority threads contend for the same locks [23].
7
In addition, multi-threaded ORBs are hard to program due tocomplex concurrency control protocols required to avoid raceconditions and deadlocks.
Solution ! the Thread-Specific Storage pattern: An ef-fective way to minimize the amount of locking required toserialize access to resources shared within an ORB is to usetheThread-Specific Storagepattern [24]. This pattern allowsmultiple threads in an ORB to use one logically global accesspoint to retrieve thread-specific data without incurring lockingoverhead for each access.
Using the Thread-Specific Storage Pattern in TAO: TAOuses the Thread-Specific Storage pattern to minimize lock con-tention and priority inversion for real-time applications. In-ternally, each thread in the TAO stores its ORB Core andObject Adapter components,e.g., Reactor , Acceptor ,Connector , POA, in thread-specific storage. When a threadaccesses any of these components, they are retrieved by us-ing akey as an index into the thread’s internal thread-specificstate, as shown in Figure 8. Therefore, no additional lockingis required to access ORB state.
THREAD ATHREAD A THREAD BTHREAD B
1: ACE_OS::thr_getspecific(key)
2: get_state(key)
ORB THREAD-SPECIFIC STATE
POA
Reactor
Acceptor
Connector
POA
Reactor
Acceptor
Connector
THREAD-SPECIFICOBJECT TABLES
INDEXED BY KEY
Figure 8: Using the Thread-Specific Storage Pattern TAO
3.6 Support Interchangeable ORB Behaviorswith the Strategy Pattern
Context: The alternative concurrency architectures de-scribed in 3.4 are just one of the many strategies that an ex-tensible ORB may be required to support. In general, exten-sible ORBs must support multiple request demultiplexing andscheduling strategies in their Object Adapters, as well as mul-tiple connection establishment, request transfer, and concur-rent request processing strategies in their ORB Cores.
Problem: One way to develop an ORB is to provide onlystatic, non-extensible strategies, which are typically config-ured in the following ways:
� Preprocessor macros: Some strategies are determinedby the value of preprocessor macros. For example, sincethreading is only available on selected platforms, conditionalcompilation is often used to select the appropriate concurrencyarchitecture.
�Command-line options: Other strategies are controlledby the presence or absence of flags on the command-line. Forinstance, command-line options can be used to enable variousORB concurrency strategies.
While these two configuration approaches are widelyused, they are very inflexible. For instance, preprocessormacros only support compile-time strategy selection, whereascommand-line options convey a limited amount of informationto an ORB. Moreover, these hard-coded configuration strate-gies are completely divorced from any code they might affect.Thus, ORB components that want to use these options must (1)know of their existence, (2) understand their range of values,and (3) provide an appropriate implementation for each value.These restrictions make it hard to develop highly extensibleORBs composed from transparently configurable strategies.
How then does an ORB (1) permit replacement of subsets ofcomponent strategies in a manner orthogonal and transparentto other ORB components and (2) encapsulate the state andbehavior of each strategy so that changes to one componentdo not permeate throughout an ORB haphazardly?
Solution! the Strategy pattern: An effective way to sup-port multiple transparently “pluggable” ORB strategies is toapply theStrategy pattern[8]. This pattern factors out simi-larity among algorithmic alternatives and explicitly associatesthe name of a strategy with its algorithm and state. Moreover,the Strategy pattern removes lexical dependencies on strategyimplementations since applications access specialized behav-iors only through common base class interfaces. In general,the Strategy pattern should be used when an application’s be-havior can be configured using multiple strategies that can beinterchanged seamlessly.
Using the Strategy Pattern in TAO: TAO uses a vari-ety of communication, concurrency, demultiplexing, real-timescheduling and dispatching strategies to factor out behaviorsthat are typically hard-coded in conventional ORBs. Severalof these strategies are illustrated in Figure 9. For instance,TAO supports multiple request demultiplexing strategies (e.g.,perfect hashing vs. active demultiplexing [25]) and schedulingstrategies (i.e., FIFO vs. rate monotonic vs. earliest deadlinefirst [14]) in its Object Adapter, as well as connection man-agement strategies (e.g., process-wide cached connections vs.
8
StrategyApplication Concurrency Scheduling Demultiplexing Protocol
Avionics Thread-per-priority Rate-based Perfect hashing VME backplaneMedical Imaging Thread-per-connection FIFO Active demultiplexing TCP/IP
Table 1: Example Applications and their ORB Strategy Configurations
Thread-Thread-SpecificSpecificConnectConnectStrategyStrategy
Cached CachedConnectConnectStrategyStrategy
Reactive ReactiveConcurrencyConcurrency
StrategyStrategy
Threaded ThreadedConcurrencyConcurrency
StrategyStrategy
SERVANT SERVANT 11 SERVANT NSERVANT N
IDLIDL
SKEL SKEL 22IDLIDL
SKEL NSKEL N
ORB COREORB CORE
SERVERSERVERCLIENTCLIENT
...... ............
SE
RV
AN
TS
ER
VA
NTN
::N
::O
PE
RA
TIO
NO
PE
RA
TIO
NKK
SE
RV
AN
TS
ER
VA
NTN
::N
::O
PE
RA
TIO
NO
PE
RA
TIO
N11
SE
RV
AN
TS
ER
VA
NT1::
1::
OP
ER
AT
ION
OP
ER
AT
ION
KK
SE
RV
AN
TS
ER
VA
NT1::
1::
OP
ER
AT
ION
OP
ER
AT
ION
22
SE
RV
AN
TS
ER
VA
NT1::
1::
OP
ER
AT
ION
OP
ER
AT
ION
11
(B) ACTIVE DEMUXING STRATEGY
index(object key)......
......
(A) PERFECT HASHING
DEMUXING
STRATEGY
OP
ER
AT
ION
OP
ER
AT
ION
KK
OP
ER
AT
ION
OP
ER
AT
ION
22
......
OP
ER
AT
ION
OP
ER
AT
ION
11
hash(operation)
hash(object key) OBJECTOBJECT
ADAPTERADAPTER
IDLIDL
SKEL SKEL 11
SERVANT SERVANT 22
ORB COREORB CORE
ORB COREORB CORE
StrategyStrategyAcceptorAcceptor
StrategyStrategyConnectorConnector
Figure 9: Strategies in TAO
thread-specific cached connections) and handler concurrencystrategies (e.g., Reactive vs. variations of Active Objects) inits ORB Core.
3.7 Consolidate ORB Strategies Using the Ab-stract Factory Pattern
Context: There are many potential strategy variants sup-ported by TAO. Table 1 shows a simple example of the strate-gies used to create two configurations of TAO. One is an avion-ics application with hard real-time requirements [14] and theother is an electronic medical imaging application [15] withhigh throughput requirements. In general, the forces that mustbe resolved to compose all ORB strategies correctly are theneed to (1) ensure the configuration of semantically compati-ble strategies and (2) simplify the management of a large num-ber of individual strategies.
Problem: An undesirable side-effect of using the Strategypattern extensively in complex software like ORBs is that ex-tensibility becomes hard to manage for the following reasons:
� Software complexity: ORB source code can becomelittered with hard-coded references to strategy types. Manyindependent strategies must act in harmony to provide a com-prehensive solution to particular application domains, such asreal-time avionics. However, identifying these strategies in-dividually by name requires tedious replacement of selectedstrategies in one domain with a potentially different set ofstrategies in another domain.
� Semantic incompatibilities: It is not always possiblefor certain ORB strategies to interact compatibly. For instance,the FIFO strategy for scheduling requests shown in Table 1might not work with the thread-per-priority concurrency archi-tecture. The problem stems from semantic incompatibilitiesbetween scheduling requests in their order of arrival,i.e., FIFOqueueing, versus dispatching requests based on their relativepriorities, i.e., preemptive priority-based thread dispatching.Moreover, some strategies are only useful when certain pre-conditions are met. For instance, the perfect hashing demul-tiplexing strategy is generally feasible only for systems thatstatically configure all servants off-line.
How can a highly-configurable ORB reduce the complexi-ties required in managing its myriad of strategies, as well asenforce semantic consistency when combining discrete strate-gies?
Solution ! the Abstract Factory pattern: An effectiveway to consolidate multiple ORB strategies into semanticallycompatible configurations is to apply theAbstract Factory pat-tern [8]. This pattern provides a single access point that inte-grates all strategies used to configure an ORB. Concrete sub-classes then aggregate semantically compatible application-specific or domain-specific strategies, which can be replacedwholesale in semantically meaningful ways. In general, theAbstract Factory pattern should be used when an applicationneeds to consolidate the configuration of many strategies, eachhaving multiple variations.
Using the Abstract Factory pattern in TAO: All of TAO’sORB strategies are consolidated into two abstract factories im-plemented as Singletons [8]. One factory encapsulates client-specific strategies, while the factory shown in Figure 10 en-capsulates server-specific strategies. These abstract factoriesencapsulate concurrency strategies in both the client and the
9
ActiveActive
DemuxingDemuxing
DispatchingDispatching
StrategyStrategy
DemuxingDemuxing
StrategyStrategy
ORBORB
ServerServer
AbstractAbstract
FactoryFactory
Concurrency Concurrency
StrategyStrategy
Perfect Perfect
HashingHashing
Avionics AvionicsConcreteConcreteFactoryFactory
Rate-based Rate-based
DispatchingDispatching
Thread- Thread-
per-per-
RateRate
Thread- Thread-
per-per-
ConnectionConnection
Medical MedicalImagingImagingConcreteConcreteFactoryFactory
FIFOFIFO
DispatchingDispatching
Figure 10: Factories used in TAO
server, and request demultiplexing, scheduling, and dispatchstrategies in the server. By using the Abstract Factory pattern,TAO can configure different ORB personalities convenientlyand consistently.
3.8 Dynamically Configure ORBs with the Ser-vice Configurator Pattern
Context: The cost of many computing resources, such asmemory and CPUs, continue to drop. However, ORBs muststill avoid excessive consumption of finite system resources.This parsimony is particularly essential for embedded real-time systems that require small memory footprints and pre-dictable CPU processing overhead. Likewise, many applica-tions can benefit from an ability to extend ORBsdynamicallyby allowing their strategies to be configured at run-time.
Problem: Although the Strategy and Abstract Factory pat-terns make it easier to customize ORBs for specific applica-tion requirements and system characteristics, these patternscan cause the following problems for extensible ORBs:
�High resource utilization: Widespread use of the Strat-egy pattern can substantially increase the number of strategiesconfigured into an ORB, which can increase the system re-sources required to run an ORB.
� Unavoidable system downtime: If strategies are con-figured statically at compile-time or static link-time using ab-stract factories, it is hard to enhance existing strategies or addnew strategies without (1) changing the existing source codefor the consumer of the strategy or the abstract factory, (2) re-compiling and relinking an ORB, and (3) restarting runningORBs and their application servants.
In general, static configuration is only feasible for a small,fixed number of strategies. Using this technique to configure
complex ORBs complicates maintenance, increases system re-source utilization, and leads to unavoidable system downtimeto add or change existing components.
How then does an ORB implementation reduce the “overly-large, overly-static” side-effect of pervasive use of the Strategyand Abstract Factory patterns?
Solution ! the Service Configurator pattern: An ef-fective way to enhance the dynamism of an ORB is toapply the Service Configurator pattern[26]. This pat-tern uses explicit dynamic linking [27] mechanisms to ob-tain, utilize, and/or remove the run-time address bind-ings of custom Strategy and Abstract Factory objectsinto an ORB at installation-time or run-time. Widelyavailable explicit dynamic linking mechanisms include thedlopen/dlsym/dlclose functions in SVR4 UNIX [28]and theLoadLibrary/GetProcAddress functions inthe WIN32 subsystem of Windows NT [29]. The ACE wrap-per facades provide a portable encapsulation of these OS func-tions.
By using the Service Configurator pattern, thebehaviorof ORB strategies are decoupled fromwhen implementa-tions of these strategies are configured into an ORB. For in-stance, ORB strategies can be linked into an ORB from DLLsat compile-time, installation-time, or even during run-time.Moreover, this pattern can reduce the memory footprint of anORB by allowing application developers to dynamically linkonly those strategies that are necessary for a specific ORB per-sonality.
In general, the Service Configurator pattern should be usedwhen (1) an application wants to configure its constituent com-ponents dynamically and (2) conventional techniques, such ascommand-line options, are insufficient due to the number ofpossibilities or the inability to anticipate the range of values.
Using the Service Configurator pattern in TAO: TAOuses the Service Configurator pattern to configure abstract fac-tories at run-time that contain the desired strategies. TAO’sinitialization code uses the dynamic linking mechanisms pro-vided by the OS and encapsulated by the ACE wrapper facadesto link in the appropriate factory for a particular use-case. Thisdesign allows applications to select the personality of a partic-ular ORB at run-time. In addition, it allows the behavior ofTAO to be tailored for specific platforms and application re-quirements without requiring access to the ORB source code.
Figure 11 shows two factories tuned for different applica-tion domains – avionics and medical imaging. In this partic-ular configuration, the avionics concrete factory is currentlyinstalled in the process. Applications using this ORB configu-ration will be processed with a particular set of ORB concur-rency, demultiplexing, and dispatching strategies. In contrast,the medical imaging concrete factory resides in a DLL outside
10
Rate-basedRate-basedDispatchingDispatching
TAOTAOPROCESSPROCESS
DLLDLLSS
Thread-per Thread-perRateRate
ConcurrencyConcurrency
AvionicsAvionicsConcreteConcreteFactoryFactory
PerfectPerfectHashingHashing
Service ServiceRepositoryRepository
ActiveActiveDemuxingDemuxing
MedicalMedicalImagingImagingConcreteConcreteFactoryFactory
FIFOFIFODispatchingDispatching
Thread-perThread-perConnectionConnection
ConcurrencyConcurrency
Figure 11: Using the Service Configurator Pattern in TAO
of the current ORB process. To support a different configu-ration of the ORB this factory could be dynamically installedwhen the ORB process is first initialized.
4 Concluding Remarks
This article presented a case study showing how we appliedpatterns to enhance the extensibility and maintainability ofTAO, a dynamically configurable, real-time ORB. We foundqualitative and quantitative evidence that the use of patternshelped to clarify the structure of, and collaboration between,components that perform key ORB tasks. These tasks includeevent demultiplexing and event handler dispatching, connec-tion establishment and initialization of application services,concurrency control, and dynamic configuration. In addition,patterns improved TAO’s performance and predictability bymaking it possible to transparently configure lightweight andoptimized strategies for processing client requests.
In general, the use of patterns in TAO provided the follow-ing benefits:
� Increased extensibility: Patterns like Abstract Factory,Strategy, and Service Configurator make it much easier to re-configure TAO for a particular application domain by allowingextensibility to be “designed into” an ORB. In contrast, mid-dleware that lacks these patterns is significantly harder to de-velop and extend. This article illustrated how design patternswere applied to make an ORB more extensible.
� Enhanced maintenance: Design patterns are essential tocapture and articulate the design rationale for complex struc-tures in an ORB. Patterns help to demystify and motivate thestructure of an ORB by describing its architecture in terms ofdesign forces that recur in many software systems. The ex-pressive power of patterns enabled us to convey the design ofcomplex software systems like TAO.
Thus, the patterns presented in this article help to improvethe maintainability of ORB middleware by reducing software
complexity.
� Increased portability and reuse: Constructing our ORBatop an OO communication framework like ACE simplifiedthe effort required to port TAO to various real-time platforms.Most of the porting effort is absorbed by the ACE frame-work maintainers. In addition, since the ACE framework isrich with configurable high-performance, real-time network-oriented components, we were able to achieve considerablecode reuse by leveraging the framework.
The use of patterns can incur some liabilities. We summa-rize these liabilities below and discuss how we minimize themin TAO.
� Abstraction penalty: Many patterns use indirection to in-crease component decoupling. For instance, the Reactor pat-tern uses virtual methods to separate the application-specificEvent Handler logic from the general-purpose event de-multiplexing and dispatching logic. The extra indirection in-troduced by using these pattern implementations can poten-tially decrease performance. To alleviate these liabilities, wecarefully applied C++ programming language features (suchas inline functions and templates) and other optimizations(such as eliminating demarshaling overhead [30] and demul-tiplexing overhead [25]) to minimize performance overhead.As a result, TAO is substantially faster than the original hard-coded SunSoft IIOP [30].
� Additional external dependencies: Whereas SunSoftIIOP only depends on system-level interfaces and libraries,TAO now depends on the ACE framework. Since ACE en-capsulates a wide range of low-level OS mechanisms, the ef-fort required to port it to a new platform could potentially behigher than porting SunSoft IIOP, which only uses a subsetof the OS’s APIs. However, since ACE has been ported tomany platforms already, the effort to port to new platforms isrelatively low. Most sources of platform variation have beenisolated to a few modules in ACE.
A final benefit of applying patterns to TAO is that not onlydid we developed a more flexible ORB, but we also deviseda richer vocabulary for discussing ORB middleware designs.This vocabulary is a key “enabling” step to demystify the in-ternals of an ORB. As we continue to learn about ORBs andthe patterns of which they are composed, we expect this vo-cabulary to grow and evolve.
The source code for ACE and TAO is freely available atwww.cs.wustl.edu/ �schmidt/TAO.html .
Acknowledgements
We would like to thank Frank Buschmann, Hans Rohnert, andMichael Stal for their extensive comments on this paper.
11
References[1] R. Johnson, “Frameworks = Patterns + Components,”Commu-
nications of the ACM, vol. 40, Oct. 1997.
[2] S. Landis and S. Maffeis, “Building Reliable Distributed Sys-tems with CORBA,”Theory and Practice of Object Systems,Apr. 1997.
[3] S. Vinoski, “CORBA: Integrating Diverse Applications WithinDistributed Heterogeneous Environments,”IEEE Communica-tions Magazine, vol. 14, February 1997.
[4] J. A. Zinky, D. E. Bakken, and R. Schantz, “Architectural Sup-port for Quality of Service for CORBA Objects,”Theory andPractice of Object Systems, vol. 3, no. 1, 1997.
[5] Object Management Group,The Common Object Request Bro-ker: Architecture and Specification, 2.2 ed., Feb. 1998.
[6] D. Box, Essential COM. Addison-Wesley, Reading, MA, 1997.
[7] A. Wollrath, R. Riggs, and J. Waldo, “A Distributed ObjectModel for the Java System,”USENIX Computing Systems,vol. 9, November/December 1996.
[8] E. Gamma, R. Helm, R. Johnson, and J. Vlissides,Design Pat-terns: Elements of Reusable Object-Oriented Software. Read-ing, MA: Addison-Wesley, 1995.
[9] D. C. Schmidt, “Experience Using Design Patterns to DevelopReuseable Object-Oriented Communication Software,”Com-munications of the ACM (Special Issue on Object-Oriented Ex-periences), vol. 38, October 1995.
[10] D. C. Schmidt, “Reactor: An Object Behavioral Pattern forConcurrent Event Demultiplexing and Event Handler Dispatch-ing,” in Pattern Languages of Program Design(J. O. Coplienand D. C. Schmidt, eds.), pp. 529–545, Reading, MA: Addison-Wesley, 1995.
[11] R. G. Lavender and D. C. Schmidt, “Active Object: an Ob-ject Behavioral Pattern for Concurrent Programming,” inPat-tern Languages of Program Design(J. O. Coplien, J. Vlissides,and N. Kerth, eds.), Reading, MA: Addison-Wesley, 1996.
[12] F. Buschmann, R. Meunier, H. Rohnert, P. Sommerlad, andM. Stal, Pattern-Oriented Software Architecture - A System ofPatterns. Wiley and Sons, 1996.
[13] D. C. Schmidt, D. L. Levine, and S. Mungee, “The Design andPerformance of Real-Time Object Request Brokers,”ComputerCommunications, vol. 21, pp. 294–324, Apr. 1998.
[14] T. H. Harrison, D. L. Levine, and D. C. Schmidt, “The De-sign and Performance of a Real-time CORBA Event Service,”in Proceedings of OOPSLA ’97, (Atlanta, GA), ACM, October1997.
[15] I. Pyarali, T. H. Harrison, and D. C. Schmidt, “Designand Performance of an Object-Oriented Framework for High-Performance Electronic Medical Imaging,”USENIX Comput-ing Systems, vol. 9, November/December 1996.
[16] V. F. Wolfe, L. C. DiPippo, R. Ginis, M. Squadrito, S. Wohlever,I. Zykh, and R. Johnston, “Real-Time CORBA,” inProceedingsof the Third IEEE Real-Time Technology and Applications Sym-posium, (Montreal, Canada), June 1997.
[17] Z. D. Dittia, G. M. Parulkar, and J. Jerome R. Cox, “The APICApproach to High Performance Network Interface Design: Pro-tected DMA and Other Techniques,” inProceedings of INFO-COM ’97, (Kobe, Japan), IEEE, April 1997.
[18] D. C. Schmidt, “ACE: an Object-Oriented Framework forDeveloping Distributed Applications,” inProceedings of the6th USENIX C++ Technical Conference, (Cambridge, Mas-
sachusetts), USENIX Association, April 1994.
[19] J. Hu, I. Pyarali, and D. C. Schmidt, “Measuring the Impactof Event Dispatching and Concurrency Models on Web ServerPerformance Over High-speed Networks,” inProceedings of the2nd Global Internet Conference, IEEE, November 1997.
[20] J. Hu, S. Mungee, and D. C. Schmidt, “Principles for Develop-ing and Measuring High-performance Web Servers over ATM,”in Proceeedings of INFOCOM ’98, March/April 1998.
[21] D. C. Schmidt, “Acceptor and Connector: Design Patterns forInitializing Communication Services,” inPattern Languages ofProgram Design(R. Martin, F. Buschmann, and D. Riehle,eds.), Reading, MA: Addison-Wesley, 1997.
[22] IEEE,Threads Extension for Portable Operating Systems (Draft10), February 1996.
[23] D. C. Schmidt, S. Mungee, S. Flores-Gaitan, and A. Gokhale,“Software Architectures for Reducing Priority Inversion andNon-determinism in Real-time Object Request Brokers,” inProceedings of the Fourth IEEE Real-Time Technology and Ap-plications Symposium, (San Francisco, CA), IEEE, December1997.
[24] D. C. Schmidt, T. Harrison, and N. Pryce, “Thread-SpecificStorage – An Object Behavioral Pattern for Accessing per-Thread State Efficiently,” inThe4th Pattern Languages of Pro-gramming Conference (Washington University technical report#WUCS-97-34), September 1997.
[25] A. Gokhale and D. C. Schmidt, “Measuring and OptimizingCORBA Latency and Scalability Over High-speed Networks,”Transactions on Computing, vol. 47, no. 4, 1998.
[26] P. Jain and D. C. Schmidt, “Service Configurator: A Patternfor Dynamic Configuration of Services,” inProceedings of the3rd Conference on Object-Oriented Technologies and Systems,
USENIX, June 1997.
[27] D. C. Schmidt and T. Suda, “An Object-Oriented Frameworkfor Dynamically Configuring Extensible Distributed Commu-nication Systems,”IEE/BCS Distributed Systems EngineeringJournal (Special Issue on Configurable Distributed Systems),vol. 2, pp. 280–293, December 1994.
[28] R. Gingell, M. Lee, X. Dang, and M. Weeks, “Shared Librariesin SunOS,” inProceedings of the Summer 1987 USENIX Tech-nical Conference, (Phoenix, Arizona), 1987.
[29] H. Custer,Inside Windows NT. Redmond, Washington: Mi-crosoft Press, 1993.
[30] A. Gokhale and D. C. Schmidt, “Principles for OptimizingCORBA Internet Inter-ORB Protocol Performance,” inHawai-ian International Conference on System Sciences, January1998.
12