dive into sobjectizer 5.5. eighth part: dispatchers

Dive into SObjectizer-5.5

SObjectizer Team, Jan 2016

Eighth Part: Dispatchers(at v.5.5.15)

This is the next part of the series of presentations with deep introduction into features of SObjectizer-5.5.

This part is dedicated to dispatchers which are one of the cornerstones of SObjectizer.

Dispatchers play a significant role in the behaviour of any SObjectizer-based application.

Because of that understanding of dispatcher concept is necessary for usage of SObjectizer framework.


https://sourceforge.net/projects/sobjectizer/


What Is A Dispatcher?




Long story short:

Dispatchers manage event queues and provide working threads on which agents handle their events.

If you don't understand what does it mean, don't worry. There is a slightly longer explanation...




The main form of agents interaction in SObjectizer is asynchronous message passing.

An agent sends a message to some mbox. If there is a subscriber for that message type on that mbox then an event is created. An event is an info about a specific message which must be delivered to the specific agent.

Event are stored in events queue.




An event must be extracted from event queue and the message from that event must be passed to agent-receiver for processing.

Agent-receiver processes message by event handler which is subscribed to this message type. Event handler is represented by agent's method or just lambda-function.

Event handler is called on the context of some thread of execution. This thread is provided by dispatcher.




Every working thread on the context of which an event handler is invoked belongs to some dispatcher.

So a dispatcher can be seen as manager of working threads to be used for invocation of event handlers.

Creation and deletion of working threads is an area of responsibility of dispatcher.




There could be many dispatchers in SObjectizer Environment. Each of them will have its own set of working threads.

Every agent should be bound to some dispatcher. Binding is done during registration of agent's coop. A programmer can select a dispatcher for a specific agent.

Once agent is bound to a dispatcher all agent's events will be handled on working threads of that dispatcher.




A dispatcher is not only a manager for working threads.

It also manages event queues for agents which are bound to that dispatcher.

It is a significant difference of SObjectizer from other actor frameworks: an event queue doesn't belong to an agent. A queue is created and managed by dispatcher.




Another difference of SObjectizer from other actor framework is a quantity of event queues.

In the "classical" actor-based approach every actor has its own queue with messages to be processed.

In SObjectizer a dispatcher makes decision how many event queues are necessary for serving agents bound to that dispatcher.




The simplest dispatcher type in SObjectizer, one_thread dispatcher, uses just one event queue for all agents bound to that dispatcher.

Much more complex dispatchers like thread_pool and adv_thread_pool can create as many event queues as it required by a programmer.

Because of that every dispatcher has its own logic of handling event queues and dispatching events for processing by agents.




So now we can say that dispatcher is responsible for:

● creation and deletion of working threads;● creation and deletion of event queues;● extraction of events from event queues and invocation of

event-handlers on context of dispatcher's working threads.

Different dispatchers do these action differently. So a user can choose the most appropriate dispatcher for a particular task.




Why Do We Need Dispatchers?




For what purpose does SObjectizer support different kinds of dispatchers and allow to create several dispatchers in an application?

Why a simple approach with one thread pool for all agents is not used?

For what reasons a user should take care about dispatcher and bind agents to some dispatcher?




It is a direct consequence of SObjectizer's primary target: simplification of development of multithreading application.

SObjectizer doesn't hide threads from user.

But simplifies their usage and provides a way for safe interaction between threads via asynchronous message passing.




A user may require dedicated threads for performing long and CPU-intensive calculations in background.

Or dedicated threads might be necessary for interaction with some hardware attached to your computer.

Or dedicated threads might be needed to do blocking calls to some 3rd party library.

Or thread pool can be necessary for efficient processing of some event stream.




So there can be different reasons to use threads for solving some real-world problems.

What we can do with that?

We can create and manage working threads manually. We can pass information between threads via some ad-hoc implementations of event queues or synchronize access to shared data via low-level mechanisms like semaphores, mutexes, condition variables and so on...




Or we can get a ready-to-use tools for management of working threads and inter-thread message passing.

SObjectizer provides such tools:

● working thread management and events scheduling is done via dispatchers;

● inter-thread message passing is implemented via general message delivery mechanism (mboxes, event subscriptions and so on).




And because different users require different thread management policies there are different dispatchers in SObjectizer.

And because of that there is a possibility to create any number of dispatchers in an application.

All of that should allow to user to solve its domain-specific task by most appropriate way.




But it doesn't mean that a user must take care about a dispatcher for every agent.

A user can take care if he wants. But if not...

...there are such things as default dispatcher and coop's main dispatcher binder which significantly simplify working with SObjectizer.




An SObjectizer Environment has the default dispatcher. It is created and started automatically.

The default dispatcher is an instance of one_thread dispatcher. It means that all agents bound to that dispatcher will work on the same working thread.

If a user doesn't specify a dispatcher for agent or agent's coop then agent will be bound to the default dispatcher.




Dispatcher Binder




A binding of an agent to a dispatcher is done via separate object called dispatcher binder.

Dispatcher binder knows how to bind agent to dispatcher instance during agent registration. And how unbind the agent during deregistration.

Every dispatcher implements its own dispatcher binder.It means that if a user wants to bind agent to active_obj dispatcher instance the user must create an active_obj dispatcher binder.




Binder can be specified for an agent during agent creation:

auto coop = env.create_coop( so_5::autoname );

// An agent will be bound to one_thread dispatcher with name "Monitor".coop->make_agent_with_binder< some_agent_type >( // Creation of dispatcher binder. so_5::disp::one_thread::create_disp_binder( "Monitor" ), ... /* Args for some_agent_type constructor */ );

// An agent will be bound to active_obj dispatcher witn name "Handler".coop->make_agent_with_binder< some_agent_type >( so_5::disp::active_obj::create_disp_binder( "Handler" ), ... );




Binder can also be specified for the whole coop. Such binder will be main binder for that coop:

auto coop = env.create_coop( so_5::autoname, // A main binder for coop will be a binder to active_obj dispatcher with the name "Handler". so_5::disp::active_obj::create_disp_binder( "Handler" ) );

// This agent will be bound via coop's main binder to active_obj dispatcher with the name "Handler".coop->make_agent< some_agent_type >( ... );

// This agent will also be bound to active_obj dispatcher with the name "Handler".coop->make_agent< another_agent_type >( ... );

// But this agent will be bound to one_thread dispatcher with the name "Monitor". It is because// a separate dispatcher binder is set for that agent.coop->make_agent_with_binder< some_agent_type >( so_5::disp::one_thread::create_disp_binder( "Monitor" ), ... );




If a dispatcher binder is not set for a coop then a binder for the default dispatcher will be used as main binder for that coop.




Public And Private Dispatchers




There are two kinds of dispatchers in SObjectizer: public and private.

There are important differences in how to work with them.




Public dispatchers must have unique names. They are accessed by means of their names.

Once created public dispatchers stay inside SObjectizer Environment until Environment finished its work.

It means that an instance of public dispatcher will work even when no one is using it.




Private dispatchers are accessible only via direct references obtained during dispatchers creation.

Private dispatchers are destroyed automatically when no one uses them.




A public dispatcher instance is passed to SObjectizer Environment via environment_params_t object which could be tuned in the launch() function:

so_5::launch( []( so_5::environment_t & ) { ... }, // Environment's parameters tuning lambda. []( so_5::environment_params_t & params ) { // Addition of a public active_obj dispatcher with the name "Handler". params.named_dispatcher( "Handler", so_5::disp::active_obj::create_disp() );

// Addition of a public one_thread dispatcher with the name "Monitor". params.named_dispatcher( "Monitor", so_5::disp::one_thread::create_disp() ); } );




Another way of creation of a public dispatcher instance is the usage of add_dispatcher_if_not_exists method:

void my_agent::evt_long_request( const request & evt ){ // New child cooperation is necessary for handling the request. // An instance of dispatcher is required for the cooperation.

// Create a dispatcher if it doesn't exist. so_environment().add_dispatcher_if_not_exists( "request_handler", [] { return so_5::disp::active_group::create_disp(); } );

so_5::introduce_child_coop( *this, so_5::disp::active_group::create_disp_binder( "request_handler" ), [&]( so_5::coop_t & coop ) { ... } );}




A binder for public dispatcher is created via appropriate create_disp_binder() function which receives the name of the dispatcher:

auto coop = env.create_coop( so_5::autoname, // A main binder for coop will be a binder to active_obj dispatcher with name "Handler". so_5::disp::active_obj::create_disp_binder( "Handler" ) );

...

// This agent will be bound to one_thread dispatcher with name "Monitor". It is because// a personal dispatcher binder is set for that agent.coop->make_agent_with_binder< some_agent_type >( so_5::disp::one_thread::create_disp_binder( "Monitor" ), ... );




An instance of private dispatcher is created by appropriate create_private_disp() which is implemented for every standard dispatcher type in SObjectizer.

A kind of smart pointer is returned by create_private_disp() function. This handle can be used for binding agents for that private dispatcher instance.

The instance will be destroyed automatically when no one uses this handle anymore.




An example of creating private dispatchers:

auto coop = env.create_coop( so_5::autoname, // A main binder for coop will be a binder to a private active_obj dispatcher. so_5::disp::active_obj::create_private_disp( env )->binder() );

// This agent will be bound via coop's main binder to active_obj dispatcher.coop->make_agent< some_agent_type >( ... );

// This agent will also be bound to active_obj dispatcher.coop->make_agent< another_agent_type >( ... );

// But this agent will be bound to private one_thread dispatcher.coop->make_agent_with_binder< some_agent_type >( so_5::disp::one_thread::create_private_disp( env )->binder(), ... );




Public dispatchers were created in the very first versions of SObjectizer and they have some limitations. For example there is no way to stop a public dispatcher if no one uses it.

Private dispatchers were added later and they solve some problems related to public dispatchers.

Because of that usage of private dispatchers is preferable.




Event Handler Thread Safety




All event handlers are treated as not thread safe by default.

It means that if there are events E1 and E2 in event queue for some agent A then these events will be delivered to A one after another.

It is impossible that event handler for E1 is running at the same time as event handler for E2 but on different thread.

SObjectizer guarantees that one event handler completed its execution and only then the next event handler will be called.




Thread safety has sense on dispatchers which use pools of threads.

Such dispatchers can migrate an agent from one working thread to another.

Because such migration can take place the SObjectizer's thread safety guarantee simplifies development of agents: there is no need for synchronization of access to agent's internals by using locks or something like that.




User can mark an event handler as thread safe.

For example if event handler doesn't modify state of the agent. Or if such modification is implemented via usage of appropriate mechanism like mutex or spinlock.

Most of standard SObjectizer's dispatchers will ignore that mark and will treat every event handler as thread unsafe.

But there is adv_thread_pool dispatcher which can use that mark appropriately.




When agent is bound to adv_thread_pool dispatcher and event handler H is marked as thread safe then the dispatcher can schedule calls of H for different events on several of working threads at the same time.

Moreover if agent has thread safe event handler H and not thread safe event handler G then adv_thread_pool will guarantee that all event handlers H will finish its work before a single event handler G will be invoked.




At v.5.5.15 there is only one dispatcher which can distinguish between not thread safe and thread safe event handlers.

But more dispatchers with such abilities can be introduced in the future versions of SObjectizer.




A Brief Overview Of Existing Dispatchers




There are eight standard dispatchers in SObjectizer-5.5

Five of them do not support agent's priorities: one_thread, active_obj, active_group, thread_pool and adv_thread_pool. These dispatcher think that all agents have the same priority.

Three dispatchers support agent's priorities: prio_one_thread::strictly_ordered, prio_one_thread::quoted_round_robin, prio_dedicated_threads::one_per_prio.They schedule agents with respect to agent's priority.




All stuff related to some dispatcher is defined in a specific namespace inside so_5::disp.

For example:

so_5::disp::active_objso_5::disp::thread_poolso_5::disp::prio_one_thread::strictly_ordered




Every dispatcher-specific namespace has the similar set of functions and classes like:

create_disp, create_private_disp, create_disp_binder

queue_params_t, disp_params_t, bind_params_t

This makes usage of different dispatchers very similar: if you know how to create one_thread dispatcher you easily create create active_obj or thread_pool dispatcher.




one_thread Dispatcher




one_thread dispatcher is the oldest, simplest and, sometimes, most useful dispatcher.

It creates just one working thread and just one event queue.

Events for all agents bound to that dispatcher are stored in single event queue.

The single working thread gets the next event from the queue and runs event handler form agent-receiver. Then gets the next event and so on.




It is possible to say that one_thread implements cooperative multitasking: if some agent hangs then all other agents bound to that dispatcher will hang too.




active_obj Dispatcher




active_obj is another very old and simple dispatcher.

But now, after addition of private dispatchers to SObjectizer, it is less useful than before.

active_obj dispatcher creates a separate working thread and separate event queue for every agent bound to that dispatcher.

It means that every agent will have its own working thread and its own event queue.




active_obj dispatcher creates a new event queue and launches a new working thread for every agent bound to it.

When agent is deregistered the dispatcher stops the working thread of this agent and deletes the corresponding event queue.

It means that if there are no agents bound to active_obj dispatcher there are no working threads nor event queues.




active_obj dispatcher is useful if it is necessary to provide a separate working thread for some agent.

For example if agent needs a dedicated context for long-running activities like interaction with hardware devices, performing blocking calls to 3rd party libraries or external systems and so on.

The similar effect can be achieved by using a private one_thread dispatcher. But sometimes usage of active_obj dispatcher requires less efforts...




An example of usage of active_obj dispatcher:

auto coop = env.create_coop( so_5::autoname, // A main binder for coop will be binder to private active_obj dispatcher. // It means that all agents of the coop without explicitly specified // binder will be bound to that active_obj dispatcher and become // active objects with own working thread. so_5::disp::active_obj::create_private_disp( env )->binder() );

for( const auto & d : devices ) // Create a separate agent for every device. coop->make_agent< device_manager >( d );

env.register_coop( std::move(coop) );




active_group Dispatcher




active_group is another very old and simple dispatcher.

But now, after addition of private dispatchers to SObjectizer, it is less useful than before and can be replaced by private one_thread dispatcher.

active_group dispatcher creates a separate working thread and separate event queue for named group of agents. It means that all agents from the same group will work on a common working thread.




active_group dispatcher creates a new working thread and a new event queue when the first agent of new group is registered.

active_group dispatcher stops group's working thread and destroy group's event queue when the last agent from the group is deregistered. It means that all resources for a group are deallocated when group lost the last member.




If a new agent is added to group which disappear earlier a new group with the same name will be created.

For example:● agents A and B is being registered as part of group G1;● new group G1 is created, new working thread is started for G1;● agents A and B are deregistered;● working thread for G1 is stopped, group G1 is deleted;● agents C and D is being registered as part of group G1;● new group G1 is created, new working thread is started for G1;




Every instance of active_group dispatcher has its own set of active groups.

A group name must be unique only inside one dispatcher instance. It means that dispatcher instance ag1 can have group G1 and dispatcher instance ag2 can have group G1 too.

These groups will be different. Lifetime of ag1.G1 and ag2.G1 will be independent from each other.




active_group dispatcher can be useful if it is necessary to bind a group of agents to the same working thread. For example if these agents performs different stages of complex requests processing.

Let's imagine a processing pipeline of three stages: requests params validation, request processing and generation of results. Every of these states can take a long time so it is better to provide a separate context to every request.

It can be done by using active_group dispatcher...




An example of binding agents to the same active group:

std::string request_id = generate_request_id( request );// Create coop with request ID used as coop name.auto coop = env.create_coop( request_id, // All agents of this coop will be part of active group on // active_group dispatcher with name "Handlers". // request_id will be used as group name. so_5::disp::active_group::create_disp_binder( "Handlers", request_id ) );

// Filling coop with agents.coop->make_agent< request_checker >( request );coop->make_agent< request_handler >( request );coop->make_agent< response_generator >( request );

env.register_coop( std::move(coop) );




After addition of private dispatcher to SObjectizer active_group dispatcher can often be replaced by private one_thread dispatcher.

But sometimes active_group dispatchers can still be useful. For example if agents for request processing cannot be created at the same time and agent for the next stage can be registered only after completion of the previous state...




An example of addition of agents from different coops to the same active group:void request_acceptor::new_request( const request_type & request ) { // A cooperation with the first stage must be created. std::string request_id = generate_request_id( request ); env.introduce_coop( request_id + "-checker", so_5::disp::active_group::create_disp_binder( "Handlers", request_id ), [&]( so_5::coop_t & coop ) { coop.make_agent< request_checker >( request ); } );}...void request_checker::check_completed() { // A cooperation with the second stage can be created. std::string request_id = generate_request_id( request ); so_environment().introduce_coop( request_id + "-handler", so_5::disp::active_group::create_disp_binder( "Handlers", request_id ), [&]( so_5::coop_t & coop ) { coop.make_agent< request_handler >( request ); } );}




thread_pool Dispatcher




thread_pool dispatcher creates a pool with several working threads.

An agent bound to thread_pool dispatcher can handle its events on any of working threads from that pool.

thread_pool dispatcher guarantees that at any given moment only single event handler of the agent might run thus using only one of the working threads. It is impossible to run event handlers e1 and e2 of agent A on different thread at the same time.




The most complicated moment in thread_pool dispatcher is event queues.

There are two important things which need to be mentioned for understanding of thread_pool dispatcher logic:

● type of FIFO for agent/coop;● max_demands_at_once parameter.




Type of FIFO determines the relationship between event handlers of agents from one coop.

Suppose there is a coop with agents A, B and C. Some agent D sends the following messages in exactly that order:● M1 to A● M2 to B● M3 to A● M4 to C● M5 to B




If agents A, B and C use cooperation FIFO then all these agents will use one common event queue. Sequence of event handler calls will look like:

1. A's event handler for M12. B's event handler for M23. A's event handler for M34. C's event handler for M45. B's event handler for M5

Exactly in that order. B's event handler for M2 will be called only when A's event handler for M1 finished its work.




Cooperation FIFO also means that agents with that FIFO type cannot work on different threads at the same time.

For example it is possible that agent A will handle M1 on thread T1. Then agent B will handle M2 on thread T2. Then agent A will handle M3 on T3.

But it is impossible that A will handle M1 on T1 and B will handle M2 on T2 at the same time.




If agents A, B and C use individual FIFO then all these agents will use different and independent event queues. It is hard to predict as a sequence of event handlers will look like. For example:

1. A's event handler for M1 on thread T12. B's event handler for M2 on thread T23. C's event handler for M4 on thread T34. B's event handler for M5 on thread T25. A's event handler for M3 on thread T1




Individual FIFO also means that agents with that FIFO type can work on different threads in parallel.

For example it is possible that agent A will handle M1 on thread T1. At the same time agent B will handle M2 on thread T2. At the same time agent C will handle M4 on thread T3.




There is another side of FIFO type: thread safety.

If agents from a coop use cooperative FIFO then they do not need to synchronize access to some shared data.

Suppose several agents from one coop use common std::map object. Because these agents cannot work on different threads at the same time they can read and modify that object without any mutexes or something like that.

But if agents use individual FIFO every shared data must be protected or completely avoided.




Parameter max_demands_at_once tells how many events from an event queue can be processed by working thread before switching to processing of another event queue.

This parameter is important because every working thread in thread_pool dispatcher works this way:● gets the next non-empty event queue;● handle at most max_demands_at_once events from it;● gets the next non-empty event queue;● ...




Suppose there are agents A, B and C from one coop with cooperation FIFO and events for messages M1, M2, ..., M5 in their event queue....




If max_demands_at_once is 4 then the following scenario might happen:● some working thread T1 gets this event queue and calls

event handlers for messages M1, ..., M4;● working thread T1 switches for handling different event

queue. Event queue for A, B, C agents holds M5;● some working thread Tn gets this event queue and calls

event handler for M5;● event queue for A, B, C agents becomes empty and Tn

switches to another event queue.




If max_demands_at_once is 1 then a working thread will handle just one event from A, B, C agent queue.

Then this queue can be handled by another working thread which will handle just one event.

Then this queue can be handled by another working thread... And so on.




A more interesting situation can be if agents A, B and C use individual FIFO.

In that case there will be independent queues for these agents:● a queue for A with M1 and M3;● a queue for B with M2 and M5;● a queue for C with M4.




If max_demands_at_once is greater than 1 then there could be the following scenario:● thread T1 handles M1 and M3 for A;● thread T2 handles M4 for C;● thread T3 handles M2 and M5 for B.

But if there are just two working threads in the pool:● thread T1 handles M1 and M3 for A;● thread T2 handles M4 for C;● thread T1 handles M2 and M5 for B.




Value of max_demands_at_once determines how often a working thread will switch from one event queue to another.

It can have a huge impact on application performance: small values of max_demands_at_once will lead to frequent queue switching and this will slow down event processing.

So large values of max_demands_at_once can speed up event processing if there is a dense flow of events.




Implementation of thread_pool dispatcher makes things yet more complex and interesting: every agent for a coop can have its own parameters for thread_pool dispatcher.

It means that agent A can have individual FIFO and max_demands_at_once=100, but agents B and C from the same coop will have cooperation FIFO and max_demands_at_once=10.

In this case two different event queues will be created: one for A and another for B and C.




This complex logic of thread_pool allows precise performance tuning for complex use cases which can be found in real-life problems.

But in the simple cases the default parameters can be used (cooperation FIFO and max_demands_at_once=4). This significantly simplifies usage of thread_pool dispatcher.

Especially when several coops are bound to the same thread_pool dispatcher instance.




An example for thread_pool dispatcher:void init( so_5::environment_t & env ){ using namespace so_5::disp::thread_pool;

env.introduce_coop( // Create an instance of thread_pool dispatcher with 3 working threads. create_private_disp( env, 3 )->binder( // All agents will use individual FIFO. // Parameter max_demands_at_once will have the default value. bind_params_t{}.fifo( fifo_t::individual ) ), []( so_5::coop_t & c ) { auto collector = c.make_agent< a_collector_t >(); auto performer = c.make_agent< a_performer_t >( collector->so_direct_mbox() ); collector->set_performer_mbox( performer->so_direct_mbox() );

c.make_agent< a_generator_t >( collector->so_direct_mbox() ); });}




adv_thread_pool Dispatcher




adv_thread_pool dispatcher is similar to thread_pool dispatcher but has two important differences:

1. It allows to invoke several thread safe event handlers for an agent at the same time on different threads.

2. As a consequence of previous point there is no max_demands_at_once parameter.




Every working thread in adv_thread_pool works this way:

● gets some non-empty event queue;● gets the first event from that queue;● checks thread safety for event handler for that event;● checks a possibility of invocation of this event handler;● if event handler can be invoked it is called;● if event handler cannot be invoked then event is returned

to event queue and thread switches to another non-empty event queue.




An event handler for an agent can be called if:

● it is not thread safe and there is no any other event handlers of that agent which are running now on one of the threads;

● or it is thread safe and there is no any not thread safe event handler which is working now on some thread.

It means that only one not thread safe event handler can be called. And next event handler can be called only after completion of previous handler. Several thread safe handlers can work at the same time.




There are two types of FIFO for adv_thread_pool: cooperation and individual. Just like in thread_pool dispatcher.

But in adv_thread_pool FIFO type has influence on checking for thread safety of event handler.

Suppose there are agents A and B from one coop and they use cooperation FIFO. Suppose also that event handlers A.e1 and B.e2 are thread safe, but event handler A.e3 is not.




If there is events e1, e2, e3, e2, e1, e3 in the event queue for A and B agents then event handlers will be called in the following sequence: A.e1 on thread T1, B.e2 on thread T2.

Only after completion of event handlers A.e1 and B.e2 event handler A.e3 will be called.

Then A.e1 on T1 and B.e2 on T2. Then waiting for completion and the call to A.e3.




If agents A and B use individual FIFO and there are events sequence {e1, e3, e1} and {e2, e2} in two event queues then there could be the following scenario:

● A.e1 on T1, then A.e3 on T1, then A.e1 on T1;● B.e2 on T2;● B.e2 on T3.




adv_thread_pool can be useful for spreading thread safe event processing on several working threads.

For example there could be cryptographer agent which performs make_signature, check_signature, encrypt_block and decrypt_block operations.

These operations are thread safe because they don't change the state of cryptographer agent. Event handler for these operations can be marked as thread safe. This allows to handle several cryptographic operations at the same time in parallel.




Subscription of thread safe event handler:

class cryptographer : public so_5::agent_t{ void on_make_signature( const make_signature & ) {...} void on_check_signature( const check_signature & ) {...} void on_encrypt_block( const encrypt_block & ) {...} void on_decrypt_block( const decrypt_block & ) {...}... virtual void so_define_agent() override { so_subscribe_self() .event( &cryptographer::on_make_signature, so_5::thread_safe ) .event( &cryptographer::on_check_signature, so_5::thread_safe ) .event( &cryptographer::on_encrypt_block, so_5::thread_safe ) .event( &cryptographer::on_decrypt_block, so_5::thread_safe ) ... }}




An agent can have thread safe and not thread safe event handlers.

For example cryptographer agent can have not thread safe event handler for message reconfigure.

In this case adv_thread_pool dispatcher guarantees that all thread safe event handlers finish their work before an event handler for reconfigure message will be started.




An example of binding agent to adv_thread_pool dispatcher:

using namespace so_5::disp::adv_thread_pool;env.introduce_coop( // All agents of new coop will work on adv_thread_pool dispatcher. // Every agent will use individual FIFO. create_private_disp( env, 4 )->binder( bind_params_t{}.fifo( fifo_t::individual ) ), []( so_5::coop_t & coop ) { coop.make_agent< cryptographer >(); ... } );




prio_one_thread::strictly_ordered Dispatcher




prio_one_thread::strictly_ordered dispatcher allows for events of high priority agents to block events of low priority agents.

It means that events queue is always strictly ordered: events for agents with high priority are placed before events for agents with lower priority.




For example if event queue is {e1(a7), e2(a6), e3(a4), e4(a4)}, where a7 means an agent with priority p7, then events will be handled in exact that order.

After handling e1 the queue will be {e2(a6), e3(a4), e4(a4)}. If e5 for a5 arrived then the queue will become {e2(a6), e5(a5), e3(a4), e4(a4)}.

It means that chronological order of events will be preserved only for events of agents with the same priority.




This dispatcher could be useful if there is a necessity of handling some messages before other messages.

For example there could be a stream of tasks represented by take_job messages. There also could be a special message for task processor’s reconfiguration: new_config message.

It could have a sense to handle new_config as soon as possible.




This can be done by two agents which are bound to single prio_one_thread::strictly_ordered dispatcher.

One agent will have priority p1 and will handle new_config message. Second agent will have priority p0 and will handle take_job.

Both agents will have common shared data (at least configuration parameters, may be something else). Dispatcher prio_one_thread::strictly_ordered guarantees that new_config will be handled as soon as processing of previous message finished.




An example of binding agents to prio_one_thread::strictly_ordered dispatcher:

namespace prio_disp = so_5::disp::prio_one_thread::strictly_ordered;env.introduce_coop( // Dispatcher instance and binder for it. prio_disp::create_private_disp( env )->binder(), []( so_5::coop_t & coop ) { // An agent with higher priority. coop.make_agent< config_manager >( so_5::prio::p1 ); // An agent with lower priority. coop.make_agent< job_performer >( so_5::prio::p0 ); } );




prio_one_thread::quoted_round_robin Dispatcher




prio_one_thread::quoted_round_robin dispatcher works on round-robin principle.

It allows to specify maximum count of events to be processed consequently for the specified priority. After processing that count of events dispatcher switches to processing events of lower priority even if there are yet more events of higher priority to be processed.




Dispatcher handles no more than Q7 events of priority p7, then no more than Q6 events of priority p6, ..., then no more than Q0 events of priority p0.

If an event of higher priority is arrived during handling a quote for lower priority no switching is performed.

For example if dispatcher handles events of priority p5 and event of priority p7 is arrived the dispatcher will continue to handle events of p5, then events of p4 (if any), ..., then events of p0 (if any). And only then events of p7.




This working scheme means that agent’s priorities treated as agent’s weight.

A programmer can set bigger quotes for more prioritized (more heavyweight) agents and that agents will receive more resources than less prioritized (less weighted) agents.




A very important detail: events of the same priority are handled in chronological order.




A dispatcher of that type can be useful, for example, if there are agents which handles clients of different types.

Some clients are VIP clients and they should receive first-class quality of service and there could be other clients with lower demands for service quality.

A high priority to agents for handling VIP-client requests can be used and a large quote for that priority can be set. All other agents will have lower priority and smaller quote. As result more requests from VIP-clients will be handled but there also will be processing of request from other clients.




An example of binding agents to prio_one_thread::quoted_round_robin dispatcher:namespace prio_disp = so_5::disp::prio_one_thread::quoted_round_robin;env.introduce_coop( // Create dispatcher and define quotes for several priorities. prio_disp::create_private_disp( env, // By default every priority will have quote for 20 events. prio_disp::quotes_t{ 20 } // Priority p7 will have different quote. .set( so_5::prio::p7, 45 ) // Priority p6 will have different quote too. .set( so_5::prio::p6, 35 ) )->binder(), []( so_5::coop_t & coop ) { coop.make_agent< vip_client_processor >( so_5::prio::p7 ); coop.make_agent< ordinal_client_processor >( so_5::prio::p6 ); coop.make_agent< free_client_processor >( so_5::prio::p0 ); } );




prio_dedicated_threads::one_per_prio Dispatcher




prio_dedicated_threads::one_per_prio dispatcher creates a single dedicated thread for every priority.

It means that events for agents with priority p7 will be handled on different thread than events for agents with, for example, priority p6.

Events of the same priority are handled in chronological order.




Because of the fact that priority is assigned to an agent at its creation and cannot be changed later an agent inside prio_dedicated_threads::one_per_prio is bound to a particular working thread and do not moved from that thread to any other thread.

This property of the dispatcher can be used for binding processing of different message types to different working threads: messages of type M1 can be processed by agent A1 with priority p1, messages of type M2 -- by agent A2 with priority p2 and so on...




Assigning different priorities for agents which handle different message type may have sense if OS-specific API for changing thread priority is used.

For example an agent with priority p7 can set higher priority for its working thread in so_evt_start() method than agent with priority p6.




An example of binding agents to prio_dedicated_threads::one_per_prio dispatcher:

namespace prio_disp = so_5::disp::prio_dedicated_threads::one_per_prio;env.introduce_coop( // An instance of dispatcher and a binder for it. prio_disp::create_private_disp( env )->binder(), []( so_5::coop_t & coop ) { coop.make_agent< m1_handler >( so_5::prio::p1 ); coop.make_agent< m2_handler >( so_5::prio::p2 ); ... } );




Some Final Words




Dispatchers play significant role in application development on top on SObjectizer-5.

It is because SObjectizer do not hide multithreading from a user. But simplifies a work with multithreading in application domains where threads are important.

Application domains are different. Even inside one domain different event scheduling policies might be needed. This is why SObjectizer provides several dispatcher types.




This presentation has no intention to be a comprehensive guide for SObjectizer's dispatcher.

It is just an attempt to explain a role of dispatchers and a short overview of standard SObjectizer's dispatcher and their features.

For more detailed information please see corresponding sections in the Project's Wiki and materials in the Project's Blog.


https://sourceforge.net/p/sobjectizer/wiki/

https://sourceforge.net/p/sobjectizer/blog/





Additional Information:

Project’s home: http://sourceforge.net/projects/sobjectizer

Documentation: http://sourceforge.net/p/sobjectizer/wiki/

Forum: http://sourceforge.net/p/sobjectizer/discussion/

Google-group: https://groups.google.com/forum/#!forum/sobjectizer

GitHub mirror: https://github.com/masterspline/SObjectizer

http://sourceforge.net/projects/sobjectizer

http://sourceforge.net/projects/sobjectizer

http://sourceforge.net/p/sobjectizer/wiki/

http://sourceforge.net/p/sobjectizer/wiki/

http://sourceforge.net/p/sobjectizer/discussion/

http://sourceforge.net/p/sobjectizer/discussion/

https://groups.google.com/forum/#!forum/sobjectizer

https://groups.google.com/forum/#!forum/sobjectizer

https://github.com/masterspline/SObjectizer

https://github.com/masterspline/SObjectizer

dive into sobjectizer 5.5. eighth part: dispatchers

Software