distributed computing with daffie
DESCRIPTION
Distributed Computing with DAFFIE. Glenn Bresnahan Boston University Scientific Computing and Visualization (http://scv.bu.edu/DAFFIE/Earlab-Oct02). What is DAFFIE?. D istributed A pplications F ramework F or I mmersive E nvironments - PowerPoint PPT PresentationTRANSCRIPT
Distributed Computing with Distributed Computing with DAFFIEDAFFIE
Glenn Bresnahan
Boston University
Scientific Computing and Visualization
(http://scv.bu.edu/DAFFIE/Earlab-Oct02)
Scientific Computing and Visualization 2
What is DAFFIE?What is DAFFIE?
Distributed Applications Framework For Immersive Environments
System for easily creating distributed and shared virtual environments
Authored by Glenn Bresnahan, Erik Brisson, Robert Putnam
Scientific Computing and Visualization 3
DAFFIE Messaging SystemDAFFIE Messaging System
Application protocol
Client servicesClient application library (API)
Message router (“event server”)
Scientific Computing and Visualization 4
DAFFIE Message routingDAFFIE Message routing
client
event event serverserver
client
clientproxyproxy
clientproxyproxy
client
client
client
client
client
Scientific Computing and Visualization 5
Application ArchitectureApplication Architecture
App threadApp thread App threadApp thread …
Services(timing, streaming, file sharing)
Message Transport
Event Messaging
Scientific Computing and Visualization 6
API - ControlAPI - Control
id = event_join(server, &nc); rc = event_leave();
id = event_id(); nc = event_nclient(); millisec = event_clock(); event_sleep(millisec);
event_register(class, species, instance);
Scientific Computing and Visualization 7
Event headers informationEvent headers information
EventType(event);EventSize(event);EventFrom(event);EventTo(event);EventSeq(event);
Scientific Computing and Visualization 8
System Events – Join & LeaveSystem Events – Join & Leave
typedef struct { EVENT_HEAD head; int id; int nmember;} EVENT_JOIN;
typedef struct { EVENT_HEAD head; int id; short detail; short exitcode;} EVENT_LEAVE
Scientific Computing and Visualization 9
System Events – IdentificationSystem Events – Identification
typedef struct { EVENT_HEAD head; int tag;} EVENT_IDENTITY;
typedef struct { EVENT_HEAD head;
char host[MAXIDSTR];char class[MAXIDSTR];char species[MAXIDSTR];char instance[MAXIDSTR];int tag;
} EVENT_IDENTIFICATION;
Scientific Computing and Visualization 10
API – SendAPI – Send
rc = event_send(to, type, &event, size);– “to” is a client id or one of the following:
• EVENT_BCAST_ALL• EVENT_BCAST_NOTME• EVENT_BCAST_SERVER• EVENT_BCAST_LOCAL
– Macros• SEND(to, type, event);• BCAST(type, event);
event_drain();
Scientific Computing and Visualization 11
API - ReceiveAPI - Receive
rc = event_receive(&event);
event_receive_type(n, type1, …); event_select(on_off, et_low, et_high);
event_wait(); event_wait_timeout(millisec);
Scientific Computing and Visualization 12
API – Receive queueAPI – Receive queue
event_scan_type(n, type1, …); rc = event_scan(&event); n = event_queued();
event_flush(flag); event_flush_sync(); event_receive_enable(on_off);
Scientific Computing and Visualization 13
API – Active messagesAPI – Active messages
event_callback(type, func);
event_add_callback(type, func);event_delete_calllback(type, func);event_protect_type(on_off, et_low,
et_high);
Scientific Computing and Visualization 14
Client ServicesClient Services
Streaming– Data chunking – Bandwidth management– Dynamic tuning
File sharing– File servers– Conditional upload/download
Scientific Computing and Visualization 15
Job Initiation & MonitoringJob Initiation & Monitoring
Rely on external authentication and job initiation (e.g. ssh, Globus)– Launch scripts (ad hoc)
• cgl-video; visible.bu.edu:vgrab -compress -process; -:vswirl -steps 9999999 -np 4; -:vbroadcast; .:vdirect -object poolanim_grp ;
– Agent initiation daemon– Logging daemon
Scientific Computing and Visualization 16
Message TransportsMessage Transports
TCP/IP– Single channel or striped
Shared memoryMyrinetIP Multicast
Scientific Computing and Visualization 17
Defining Upper Layer ProtocolDefining Upper Layer Protocol
Components– Event type ID (#define in a .h)– Data layout (compile-time) (C struct in a .h) – Data layout (run-time) (event_define)• Needed for canonical byte ordering
Types statically (compile time) defined
Protocol compiler
Scientific Computing and Visualization 18
Message RoutingMessage Routing
Message router (“event server”)– Multiple delivery service classes
• Globally sequenced, reliable delivery
– Client management– Message subscription– Multiprocessor, high bandwidth
Hierarchical servers Proxy routers
– Fan-in/fan-out Tunneling
– Direct routing for P2P
Scientific Computing and Visualization 19
Messaging APIMessaging API
Multithreaded, non-blocking Dynamic join and leave Point-to-point and broadcast messages Polled or callback (active messages) Subscription based receive Low level services– Registration, timing, synchronization, query
Scientific Computing and Visualization 20
Service DiscoveryService Discovery
Client registration/identification– Class (Viewer, Audio, Video, Agent)– Species (video: source, broadcaster, sink,
processor)– Instance (video: stream-name)– ID– Hostname
Client location services Service request/acknowledgement
Scientific Computing and Visualization 21
Client MonitoringClient Monitoring
Dynamic clients– JOIN, EXIT, IDENTIFICATION events
Event server monitoring– Heart beat– Latency monitor– Congestion control– Statistic generation
Client monitoring– Process level monitoring– Congestion management– Remote process termination
Scientific Computing and Visualization 22
Leaky Example: HeadersLeaky Example: Headers
typedef struct {EVENT_HEAD head;char filename[200];
} EVENT_GETPARAM;
typedef struct {EVENT_HEAD head;int logger, frame_size, sample_rate);
} EVENT_START;
typedef struct {EVENT_HEAD head;char msg[1000];
} EVENT_LOGINFO;
Scientific Computing and Visualization 23
Leaky Example: HeadersLeaky Example: Headers
typedef struct {
EVENT_HEAD head;
int client;
int size;
float pressure[1];
} EVENT_ADVANCE;
Scientific Computing and Visualization 24
Leaky Example: HeadersLeaky Example: Headers
#define ET_START 1001#define ET_GETPARAM 1002#define ET_LOGINFO 1003#define ET_ADVANCE 1004#define ET_STOP 1005
event_define(ET_START, “start”, 1, 3, EVENT_DATA_INT);event_define(ET_GETPARAM, “getparam”, 1, 200, EVENT_DATA_CHAR);event_define(ET_ADVANCE, “advance”, 3,
1, EVENT_DATA_INT,1, EVENT_DATA_INT,0, EVENT_DATA_FLOAT);
Scientific Computing and Visualization 25
Leaky Example: InitializationLeaky Example: Initialization
id = event_join(“earlab.bu.edu”, &nc);printf(“Joined as client id %d of %d\n”, id, nc);
event_register(“integrator”, “leaky”, “1.0”);
event_receive_type(ET_GETPARAM, ET_START, ET_ADVANCE, ET_STOP);
event_callback(ET_START, do_start);event_callback(ET_GETPARAM, do_getparam);…
Scientific Computing and Visualization 26
Leaky Example: File ServiceLeaky Example: File Service
GotParamFile = 0;
efs_init();
efs_chdir_receive(“/tmp/earlab”);
Scientific Computing and Visualization 27
Leaky Example: GetParametersLeaky Example: GetParameters
do_getparams(EVENT_GETPARM *egp) {
efs_download_cond(egp->filename,
egp->filename, xfer_done, 0);
return(1);
}
xfer_done(int reqid, int fsid, inr rc, char *file, int tag) {
char path[1000];
strcpy(path, “/tmp/earlab/”); strcat(path, file);
GotParamFile = 1;
Leaky_GetParameters(path);
);
Scientific Computing and Visualization 28
Leaky Example: StartLeaky Example: Start
do_start(EVENT_START *est) {
Logger = est->logger;
while (!GotParamFile) event_sleep(100);
Leaky_Start(egp->frame_size, egp->sample_rate);
return(1);
}
Scientific Computing and Visualization 29
Leaky Example: LogInfoLeaky Example: LogInfo
void LogInfo(char *msg) {
EVENT_LOGINFO einfo;
strcpy(einfo->msg, msg);
event_send(Logger, ET_LOGINFO, &einfo, sizeof(einfo));
}
Scientific Computing and Visualization 30
Leaky Example: AdvanceLeaky Example: Advance
do_advance(EVENT_ADVANCE *ein) {int bytes;EVENT_ADVANCE *eout;
bytes = sizeof(EVENT_ADVANCE) + (ein->size-1)*sizeof(float);
eout = (EVENT_ADVANCE *) malloc(bytes);
Leaky_Advance(&ein->pressure, &eout->pressure);eout->client = event_clientid();eout->size = ein->size;event_send(ein->client, ET_ADVANCE, eout, bytes);
free(ein);return(1);
}
Scientific Computing and Visualization 31
DAFFIE Message routingDAFFIE Message routing
client
event event serverserver
client
clientproxyproxy
clientproxyproxy
client
client
client
client
client