a restful api for controlling dynamic streaming topologies
Post on 11-May-2015
198 Views
Preview:
DESCRIPTION
TRANSCRIPT
A RESTful API for Controlling Dynamic Streaming TopologiesMasiar Babazadeh!University of Lugano Switzerland @masiarb
Cesare Pautasso!University of Lugano Switzerland @pautasso
1
2
F
CP
F
F
3
P
4
F
5
C
6
P C
7
8
Web Liquid Streams
8
Web Liquid Streams
8
Web Liquid Streams
9
9
9
9
9
9
9
10
x
11
F
CP
F
F
WebSocket
WebRTC
WebSocket
ZeroMQ
ZeroMQ
x
11
F
CP
F
F
WebSocket
WebRTC
WebSocket
ZeroMQ
ZeroMQ
x
11
F
CP
F
F
WebSocket
WebRTC
WebSocket
ZeroMQ
ZeroMQ
12
F
Workers
13
F
Operator Elasticity
13
F
Operator Elasticity
13
F
Operator Elasticity
13
F
Operator Elasticity
13
F
Operator Elasticity
13
F
Operator Elasticity
13
F
Operator Elasticity
14
F
Fault Tolerance
14
F
Fault Tolerance
14
F
Fault Tolerance
14
F
Fault Tolerance
15
F
CP
F
FX
15
F
CP
F
FX
15
F
CP
F
15
F
CP
F
F
16
Operator MigrationF
CP
F
16
Operator MigrationF C
P
F
17
Peer OverloadingF
CP
F
17
Peer OverloadingF
CP
F
17
Peer OverloadingF
CP
F C
17
Peer OverloadingF
CP
F C
17
Peer OverloadingF
CP
F C
17
Peer OverloadingF
CP
F C
18
API
WLS Runtime
P C
19
RPC
20
REST
21
Operator
Tight Coupling
22
Operator
22
Operator
22
Operator
22
Operator
22
Operator
23
Independent Evolution
Operator
23
Independent Evolution
Operator
23
Independent Evolution
Operator
24
Operator
Error Detection & Recovery
24
Operator
Error Detection & Recovery
operators[0].getWorkers();
24
Operator
Error Detection & Recovery
Exception
24
Operator
Error Detection & Recovery
GET …/workers
24
Operator
Error Detection & Recovery
“request timeout”
24
Operator
Error Detection & Recovery
{!! “error” : …!}
24
Operator
Error Detection & Recovery
GET …/workers
24
Operator
Error Detection & Recovery
25
Representations
{ "topology": { "id": "example", "operators": […], "bindings": […] } }
<topology> <id>example</id> <operators> <operator>…</operator> … </operators> <bindings> <binding>…</binding> … </bindings> </topology>
2626
F
CP
F
F
2626
F
CP
F
F
Operators
2626
F
CP
F
F
Bindings
2626
F
CP
F
F
Workers
2626
F
CP
F
F
Peers
2626
F
CP
F
F
Topologies
27
Hypermedia Data Model
28
Peer Discovery
C
F GET /!GET /peers!POST /peers!GET /peers/:pid!DELETE /peers/:pid
29
Peer Representation
{! "peer": ! {!!!!!!"id": "example",! !!!!!"operators": […],! !!!!!"topologies": […],! !!!!!"CPU": [0, 50, 0] ! !!!}! }
GET /peers/example
30
Topologies
GET /topologies!POST /topologies!GET /topologies/:tid!PUT /topologies/:tid!DELETE /topologies/:tid
F
CP
F
31
Topology Representation
{ "topology": { "id": "example", "operators": […], "bindings": […] } }
GET /topologies/example
32
F
Operators
GET /operators!GET /topologies/:tid/operators!GET /topologies/:tid/operators/:oid!PUT /topologies/:tid/operators/:oid!PATCH /topologies/:tid/operators/:oid/script!PATCH /topologies/:tid/operators/:oid/bindings!DELETE /topologies/:tid/operators/:oid
33
{! "operators" : [! {! "topology" : "example",! "id" : "prod",! "workers" : […],! "href" : “…/topologies/example/operators/prod”,! "peer" : "http://IP:port/",! "replicas" : [! ! "http://IP2:port2/topologies/example/operators/prod"!! ! ] !
! },!
]!
}
Operator RepresentationGET /topologies/example/operators
34
“topology” : { “operators: […], "bindings": [ { "from": “…/topologies/example/operators/a”, "to": “…/topologies/example/operators/b”, "type": "round_robin" }, { "from": “…/topologies/example/operators/b”, "to": “…/topologies/example/operators/c”, "type": "broadcast" } ] }
Binding Representation
35
Workers
GET !/topologies/:tid/operators/:oid/workers!GET !/topologies/:tid/operators/:oid/workers/:wid!POST !/topologies/:tid/operators/:oid/workers!DELETE !/topologies/:tid/operators/:oid/workers/:wid
36
“operators“ : [!
!{!
! "topology" : "example",!
"id" : "prod", ! "workers" : [!! ! {!! ! ! "id" : "0",!! ! ! "href" : “…/topologies/example/operators/prod/workers/0”,!! ! ! "operator" : “…/topologies/example/operators/prod”,!! ! ! "uptime" : 3600,!! ! ! "messages" : 42,!! ! ! "req-res-ratio" : 1.5!! ! }, …!
! ],!
! …!
! }!
]
Worker Representation
37
3838
F
Load Balancing
3838
F
Load Balancing
3838
F
GET /topologies/:tid/operators/:oid/workers
Load Balancing
3838
F
“workers”: [ ! {!! ! ! "id" : "0",!! ! ! "href" : …,,!! ! ! "operator" : …,!! ! ! "uptime" : 3600,!! ! ! "messages" : 42,!! ! ! "req-res-ratio" : 1.5”,!! ! },! !…! ]Load Balancing
3838
F
POST /topologies/:tid/operators/:oid/workers
Load Balancing
3838
F
Load Balancing
3838
F
Load Balancing
3939
F
Fault Tolerance
3939
F
Fault Tolerance
3939
F
GET /topologies/:tid/operators/:oid/workers
Fault Tolerance
3939
F
“workers”: [ ! {!
! ! ! "id" : "0",!! ! ! "href" : “…/prod/workers/0”,!! ! ! …!! ! },!
{!
! ! ! “id” : “1”,!! ! ! “href” : …,!! ! ! “error”: “request timeout”!
! ! }!
]Fault Tolerance
3939
F
POST /topologies/:tid/operators/:oid/workers
Fault Tolerance
3939
F
Fault Tolerance
3939
F
DELETE /topologies/:tid/operators/:oid/workers/:wid
Fault Tolerance
3939
F
Fault Tolerance
40
F
CP
F
Fault Tolerance
40
F
CP
F
Fault Tolerance
40
F
CP
F
GET /topologies/:tid/operators
Fault Tolerance
40
F
CP
F
“operators”: [ ! {!! ! ! …!! ! },! {!! ! “id” : “C”,!! ! “href” : …,!! ! “error” : !! ! “request timeout”!! ! }! ]Fault Tolerance
40
F
CP
F
PUT /topologies/:tid/operators/C
Fault Tolerance
PATCH /topologies/:tid/operators/bindings
40
F
CP
F
C
Fault Tolerance
40
F
CP
F
C
DELETE /topologies/:tid!/operators/C
Fault Tolerance
40
F
P
F
C
Fault Tolerance
41
F
CP
F
Fault Tolerance
41
F
CP
F
Fault Tolerance
41
F
CP
F
GET /peers/:pid
Fault Tolerance
connection timeout
41
F
CP
F
Fault Tolerance
PUT /topologies/:tid/operators/…
41
F
CP
F
Fault Tolerance
41
F
CP
F
CF
Fault Tolerance
PATCH /topologies/:tid/operators/bindings
41
F
CP
F
CF
Fault Tolerance
41
F
CP
F
CF
Fault Tolerance
41
F
CP
F
CF
Fault Tolerance
DELETE /topologies/:tid!/operators/…
41
F
CP
F
CF
Fault Tolerance
41
F
P
CF
Fault Tolerance
42
F
CP
F
Peer Overloading
42
F
CP
F
GET /peers/:pid
Peer Overloading
42
F
CP
F
{ !
! “peer”: ! {!
! ! ! …!
! ! ! “CPU” : !
! ! ! [100,100,100]! !
! }!
}Peer Overloading
PUT /topologies/:tid/operators/C
42
F
CP
F
Peer Overloading
PATCH /topologies/:tid/operators/bindings
42
F
CP
F
C
Peer Overloading
42
F
CP
F
C
Peer Overloading
42
F
CP
F
C
Peer Overloading
43
F
CP
F
Leaving Peer
43
F
CP
F
DELETE /peers/IP:port
Leaving Peer
43
F
CP
F
Leaving Peer
43
F
CP
F
PUT /topologies/:tid/operators/…
Leaving Peer
43
F
CP
F
CF
PATCH /topologies/:tid/operators/bindings
Leaving Peer
43
F
CP
F
CF
DELETE /topologies/:tid!/operators/…
Leaving Peer
43
F
P
CF
Leaving Peer
43
F
P
CF
Leaving Peer
44
44
44
44
44
44
A RESTful API for Controlling Dynamic Streaming TopologiesMasiar Babazadeh!University of Lugano Switzerland @masiarb
Cesare Pautasso!University of Lugano Switzerland @pautasso
45
http://design.inf.usi.ch/
top related