xmpp/jingle(voip)/perl ocean 2012/03
DESCRIPTION
Perl Oceanを作成したときの資料 XMPP/Jingleなどのプロトコルの説明や、古いブラウザやWebSocketのような新技術とのインテグレーションについてTRANSCRIPT
![Page 1: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/1.jpg)
XMPP Based Realtime Communication
Framework Suite
!
加藤 亮 2012/03
Ocean
![Page 2: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/2.jpg)
Realtime
• Data (micro contents)
• Delivery Method ( Push )
• UI
![Page 3: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/3.jpg)
PUSH Friendly Contents
• Notification
• Activity Stream
• Message, Chat
• Presence
![Page 4: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/4.jpg)
PUSH
• Long Polling
• XMPP
• HTML5 (SSE, WebSocekt)
• Smart phone (APNS, C2DM)
![Page 5: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/5.jpg)
UI
• Ajaxify - auto view updating
![Page 6: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/6.jpg)
Ocean
![Page 7: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/7.jpg)
Framework Suite
• Front Server
• Cluster
• HTTP Binding
• P2P(Audio/Video chat) 予定
![Page 8: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/8.jpg)
Protocol
• XMPP Based
• XMPPのサブセットをサポート
• XMPPと内部で互換性を持たせた数種類の独自プロトコル
• External WebSocket Binding
• HTTP Binding ( WebSocket/SSE/XHR )
![Page 9: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/9.jpg)
Jabber/XMPP
![Page 10: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/10.jpg)
examples
• google talk
• facebook chat
• PSN
• iMessage
• google wave ( S2S )
![Page 11: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/11.jpg)
Stream Procedure
• Stream Initialization
• TLS Negotiation
• Authentication
• Resource Binding
• Session Establishment
• Roster Request
• Initial Presence
![Page 12: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/12.jpg)
Stanza
• Feature
• Auth
• Message
• Presence
• IQ
![Page 13: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/13.jpg)
ID
• JID (example: [email protected]/resource )
• Full JID ([email protected]/resource)
• Bare JID ([email protected])
![Page 14: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/14.jpg)
Authentication• SASL
• DIGEST-MD5
• PLAIN ( under TLS )
• EXTERNAL ( for S2S TLS )
• We need
• OAuth 2.0?
• Cookie/Ticket ( for WebSocket )
![Page 15: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/15.jpg)
Message
![Page 16: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/16.jpg)
Message
• service notification (type=headline)
• chat message (type=chat)
• group chat message (type=groupchat)
• normal ? message (type=message)
![Page 17: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/17.jpg)
Presence
![Page 18: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/18.jpg)
Presence
• presence
• subscription
• Ocean doesn’t support subscription
• extension
• join/leave a room
![Page 19: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/19.jpg)
Presence
• show (chat, away, dnd, xa )
• status (custom message)
• priority
![Page 20: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/20.jpg)
Presence traps
• protocol design
• presence:connection = 1:1
• de facto usage
• presence:user = 1:1
![Page 21: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/21.jpg)
Presence De Facto Usage
• one presence for one user
• we don’t need priority
• ‘chat’ and ‘away’ are enough for ‘show’ param
![Page 22: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/22.jpg)
Resource Binding
• node@domain/resource
• you can choose ‘home’ or ‘work’
• de facto usage
• users and client softwares don’t mind what the resource is.
• so server generates random string (like a cookie )
![Page 23: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/23.jpg)
Session Establishment
• Historical Reason
• Do nothing
![Page 24: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/24.jpg)
IQ
• Roster ( Friend List )
• vCard ( Profile )
• bind, session, ping, disco, etc...
• packets should be back to sender
![Page 25: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/25.jpg)
Roster & vCard• like OpenSocial’s ‘People Data API’
![Page 26: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/26.jpg)
Roster
• there isn’t profile image, which is included in vCard
• no pagination control by default
![Page 27: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/27.jpg)
vCard traps
• just for profile image
• but base64 encoded binary
• we want URL
![Page 28: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/28.jpg)
Subscription & Roster Push
• Friend Request
• Relationship State Control
![Page 29: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/29.jpg)
C2S & S2S
• Client to Server
• Server to Server
• Ocean doesn’t support S2S
![Page 30: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/30.jpg)
C2S Example
gmail
User A User B
C2S StreamC2S Stream
![Page 31: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/31.jpg)
S2S Example
gmail.com
User A User B
chat.facebook.com
User DUser C
S2S Stream
![Page 32: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/32.jpg)
S2S Problems
• We have to manage new database that represents ‘relationships’ between users of our service and ones of other service.
• We have to manage a white list or spam control. ( like OpenID )
• Authentication Compatibility Cost
• Scaling Cost
• Error Handling
![Page 33: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/33.jpg)
Implementation
![Page 34: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/34.jpg)
Other Implementations• I/O Multiplexing
• Node.js
• Apache MINA, naggati
• Jabber/XMPP software
• ejebberd
• DJabberd
• Openfire
![Page 35: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/35.jpg)
Ocean’s XMPP support
• subset
• customized
• fixed features(not pluggable )
• assumed cluster installation
![Page 36: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/36.jpg)
Why subset?
• Ocean is just for Web-based Social Network Service
• Create/Modify friendships on the web site.
• S2S costs too much
• Developers have to write only minimum number of event handlers.
![Page 37: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/37.jpg)
Protocol Customized
• resource binding & roster/vCard responses support extra-attributes.
• Because of regardless of HTTP federation, especially URL
• WebSocket binding (experimental )
• try demo application
• Simple-MUC
![Page 38: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/38.jpg)
Not flexible and pluggable?
• It may cause complication. requires detailed protocol knowledge, framework knowledge.
• When the servers are distributed, front-server-plugins require complicated spec. See ‘Cluster’ section.
• There are de-facto-standard specs.
![Page 39: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/39.jpg)
Why full scratch?
• Proprietary Reason
• We may need non-standard features
• Rapid Updating (community based development requires many arguments)
• Architectural Dependencies
• No Money
![Page 40: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/40.jpg)
Inside Ocean
![Page 41: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/41.jpg)
Getting Started
• All you need to do is ...
• generate project template
• edit configuration file
• prepare your handler class
![Page 42: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/42.jpg)
Generate Project Template
• ./ocean
• cd Foobar
![Page 43: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/43.jpg)
Configuration
• vi conf/ocean.yml
![Page 44: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/44.jpg)
Write Handler Class
• inherit Ocean::Handler
• implement all event handler methods
![Page 45: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/45.jpg)
Ocean Components
Server Stream Manager
Stream Stream Stream
Handler
Daemonizer Signal Handler Listener Stream Builder
![Page 46: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/46.jpg)
Stream Components
IO Components
Inside Stream
Stream Manager
Encoder
Decoder
Socket
Protocol States
Client
SASL
Active
...
ServerHandler
Events
![Page 47: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/47.jpg)
Problem?
• Bad performance
• too many call stack on each event
• each component consume memory
![Page 48: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/48.jpg)
Benefit
• Testability
• You can switch to stub components
• Customizability
• You can switch Encoder/Decoder
![Page 49: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/49.jpg)
Pay attention to memory usage
• 4byte on a Stream object
• 4x10,000byte on a Server witch has 10,000 connection
• bless Array instead of Hash
• use Config instance as singleton object
• use delegate object for event handling
![Page 50: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/50.jpg)
Your Service
Databases
Handler
Handler
Users RelationshipsConnection Info
User A Home chat
User A Work away
User B Home chat
User C Home chat
Files
Profile Pictures
![Page 51: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/51.jpg)
Handler uses 2 or 3 kinds of store
• Persistent Data Store (users, relationships)
• Half Persistent Data Store. Same lifecycle as XMPP service (connection map)
• Cache system ( if needed )
![Page 52: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/52.jpg)
Handler Implementation
![Page 53: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/53.jpg)
Handler Example
• modules under Ocean::Standalone::Handler namespace are good example
![Page 54: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/54.jpg)
Server Events
• on_node_init
• on_node_exit
![Page 55: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/55.jpg)
Stream Events
• on_sasl_auth_request
![Page 56: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/56.jpg)
Stream Events
• on_bind_request
![Page 57: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/57.jpg)
Stream Events
• on_message
![Page 58: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/58.jpg)
Stream Events
• on_presence
• on_initial_presence
• on_unavailable_presence
![Page 59: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/59.jpg)
Presence doesn’t scale• user_001 login-ed ( send initial-presence )
• get all friends of user_001
• get all available-resources of the friends
• send user_001’s presence to all the available-resources.
• send presences of the all available-resources to user_001
It’s important to limit the maximum number of friends.
![Page 60: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/60.jpg)
Stream Events
• on_silent_disconnection
![Page 61: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/61.jpg)
Stream Events
• on_oster_request
![Page 62: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/62.jpg)
Stream Events
• on_vcard_request
![Page 63: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/63.jpg)
Stream Events
• on_too_many_auth_attempt
![Page 64: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/64.jpg)
Start & Stop
• bin/ocean-start
• bin/ocean-start --daemonize
• bin/ocean-stop ( if daemonized )
![Page 65: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/65.jpg)
Cluster
• C10K Problem on Persistent Connection
• Blocking IO multiplexing
• Message delivery/Presence broadcast
![Page 66: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/66.jpg)
Message Delivery
XMPP Server 1
User A (Home)
User B (Home)
XMPP Server 2
User C (Home)
Delivery Server
Connection Map
User A Home 1
User A Work 2
User B Home 1
User C Home 2
User A (Work)
![Page 67: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/67.jpg)
Use 2 kinds of gearman
• Background Job Dispatcher
• Inbox ( 1 inbox for each front-server )
![Page 68: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/68.jpg)
Message Delivery
XMPP Server 1 XMPP Server 2
Delivery Server
Connection Map
User A Home 1
User A Work 2
User B Home 1
User C Home 2
Background Job Dispatcher (gearmand)
inbox1 (gearmand)
inbox2 (gearmand)
![Page 69: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/69.jpg)
Message Delivery
XMPP Server 1 XMPP Server 2
Delivery Server
Connection Map
User A Home 1
User A Work 2
User B Home 1
User C Home 2
Background Job Dispatcher (gearmand)
inbox1 (gearmand)
inbox2 (gearmand)
![Page 70: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/70.jpg)
Message Delivery
XMPP Server 1 XMPP Server 2
Delivery Server
Connection Map
User A Home 1
User A Work 2
User B Home 1
User C Home 2
Background Job Dispatcher (gearmand)
inbox1 (gearmand)
inbox2 (gearmand)
![Page 71: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/71.jpg)
Connection Map• too many ‘read’
• ‘write’ on resource-binding, presence-change
• Implementation
• TokyoTyrant
• HandlerSocket
• must not be volatile
![Page 72: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/72.jpg)
Ocean::Cluster
• Ocean::Cluster::Gearman::Server
• Ocean::Cluster::Gearman::Worker
![Page 73: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/73.jpg)
Cluster support on front server
• set config ‘handler_class=Ocean::Cluster::Gearman::Server::Handler’
• set inbox and background-job-dispatcher
![Page 74: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/74.jpg)
Front server components
XMPP Server
inbox (gearmand)
Background Job Dispatcher (gearmand)
Ocean::Cluster::Gearman::Server::Handler
![Page 75: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/75.jpg)
Build worker project
!
• same as front server
• build project-template
• edit configuration
• write your handler class
![Page 76: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/76.jpg)
Generate Project Template
• ocean_cluster_gen --project=FooWorker --handler=FooWorker::Handler
• cd FooWorker
![Page 77: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/77.jpg)
Configuration
• set information about all your inboxes
• set information about background-job-dispatcher
![Page 78: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/78.jpg)
Worker
Worker side (delivery server) components
inbox0 (gearmand)
Background Job Dispatcher (gearmand)
inbox1 (gearmand)
Handler
Serializer
Subscriber Deliverer
Process Manager
Connection Map
User A Home 1
User A Work 2
User B Home 1
User C Home 2
DB (users, relationships, etc...)
![Page 79: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/79.jpg)
Transaction
• 同じユーザーからのイベント順序が壊れないように調整が必要
![Page 80: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/80.jpg)
Operation
![Page 81: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/81.jpg)
Balancing
• Round-Robin
• DNS SRV Record
• _xmpp_client._tcp.chat.mixi.jp 86400 IN SRV 10 20 5222 chat1.mixi.jp
• _xmpp_client._tcp.chat.mixi.jp 86400 IN SRV 10 5 5222 chat2.mixi.jp
![Page 82: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/82.jpg)
chat1.mixi.jp
User A (Home)
Service Status Store
name connections wait
chat1 100000 2
chat2 1000 1
chat3 100000 2
chat2.mixi.jp chat3.mixi.jp
Update Periodically
Status API
Service Status API and Client Hack
![Page 83: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/83.jpg)
chat1.mixi.jp
User A (Home)
Service Status Store
name connections wait
chat1 100000 2
chat2 1000 1
chat3 100000 2
chat2.mixi.jp chat3.mixi.jp
Update Periodically
Status API
Service Status API and Client Hack
![Page 84: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/84.jpg)
chat1.mixi.jp
User A (Home)
Service Status Store
name connections wait
chat1 100000 2
chat2 1000 1
chat3 100000 2
chat2.mixi.jp chat3.mixi.jp
Update Periodically
Status API
Service Status API and Client Hack
![Page 85: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/85.jpg)
Update & Reboot
• Worker side handler includes service-code, so it’s required updating and rebooting frequently. Worker’s job is stateless. So, it’s no problem.
• Front-end server doesn’t include service-code. So you don’t need to reboot it frequently, but when you want to do it, you have to pay attention.
![Page 86: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/86.jpg)
Shutdown• You can’t shutdown server without client
awareness.
• You have to write re-connection on client side code
• Problem: algorithm distribution case
• At graceful shutdown, unavailable presences are handled correctly
• Problem: at non-graceful shutdown
![Page 87: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/87.jpg)
Front-server Broken
• フロントがぶっ壊れたときのために、ConnectionMapから、壊れたサーバーにつないでたユーザー全員のunavailable
presenceを処理するバッチを用意する必要がある
![Page 88: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/88.jpg)
Front-server maintainance
• algorithm分散のフロントを定期再起動するときは、代替が先に必要
• group_id = get_group(user_id)
• host = get_host(group_id)
• 代替サーバーを準備し、group/host
マップを書き換えてから、shutdown
![Page 89: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/89.jpg)
Service(Client-side) Strategy
![Page 90: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/90.jpg)
Device/Platform
• PC Web ( HTTP Binding )
• PC Desktop App(AIR, Titanium, or Native)
• Smartphone
• Tablet
• Browser extension
![Page 91: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/91.jpg)
Service
• 1to1 Messaging
• Room? (group chat)
• Voice/Video chat
• Push Notification
![Page 92: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/92.jpg)
Ocean supports
• WebSocket hybi-draft-10 support
• HTTPBinding(WebSocket/SSE/XHR)
![Page 93: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/93.jpg)
HTTP Binding
![Page 94: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/94.jpg)
Methods
• Polling ( Ajax ) + Post API (+APNS|C2DM)
• Long Polling ( Ajax, Comet ) + Post API
• ServerSentEvent + Post API
• WebSocket
• TCP XMPP
![Page 95: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/95.jpg)
‘Stateless’ problems
• Different way of connection management from ordinary persistent connection
• single user-session may open many tabs
• user moves in short time. too many re-connection
• how to detect user to leave? (leave or move)
![Page 96: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/96.jpg)
server 1
User A
server 2 server 3
Connection Map
User A Home 1
User A Work 2
User B Home 1
User C Home 2
too many ‘write’ and ‘presence’ in short time
open many tabs or hop http-link in short time
TabTab
Tab
mod_proxy(_balancer)
User B
when user leaves?
![Page 97: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/97.jpg)
Solution 1 Algorithm Based Distribution
• ユーザー毎の宛先を固定することによって、Connection Mapを利用しなくても、特定のユーザーにパケットを届けることは可能になる
• ユーザーがWebサービスを開いてるかどうかは関係なくなる
![Page 98: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/98.jpg)
Problem
• Connection Mapを利用しないと、正確なプレゼンスを把握できなくなる
![Page 99: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/99.jpg)
Solution 2. Stream Aggregation
• 同一cookieを持つ接続をまとめあげるSessionオブジェクトを利用する
• 状態管理はSessionオブジェクトで工夫
• Pending TimerでLeaveの検知
![Page 100: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/100.jpg)
User B
hb1.mixi.jp hb2.mixi.jp hb3.mixi.jp
User A
TabTab
TabUser C
TabTab
Tab
Connection Map
User A Home 1
User A Work 2
User B Home 1
User C Home 2Aggregate
connections as single session
algorithm based distribution: server_node=algorithm(user_id)
Manager Manager Manager
use timer to detect users to leave.
(*) number of simultaneous Ajax request directed to same domain is limited, so we need many nickname for each hb servers. (if we adopt long polling)
until the expiration, server should buffer messages if received
![Page 101: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/101.jpg)
Server Stream Manager
User A Stream
User A Stream
Handler
Session
Handle Multiple Streams as One Single Session
User B Stream
User B Stream
Session
Auth
ResourceBinding
SessionEstablishment
InitialPresence
Handshake
![Page 102: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/102.jpg)
Tasks
• We need to
• prepare social ad system
• improve UI (moveless)
![Page 103: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/103.jpg)
PubSub Support
• XEP-0061 Publish Subscribe
• We don’t need full-spec support
• don’t allow use to publish/subscribe
• server publish event
![Page 104: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/104.jpg)
Publish Event from Web Service
XMPP Server 1
User A (Home)
User B (Home)
XMPP Server 2
User C (Home)
Delivery Server
User A (Work)
Web ServiceUserEvents
(Notification, Activity)
![Page 105: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/105.jpg)
XMPP (HTTPBinding) Streams
Web Service
UserEvents (Notification,
Activity)
APNSC2DM EmailWebHook PubsubHubbub
DeliveryCondition: ContentKind, UserPresence
![Page 106: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/106.jpg)
Groupchat
![Page 107: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/107.jpg)
Overview
• XMPP has MUC extension spec. but it’s over spec, so we need arrangement
• Costs too much like presence
• limit numbers of channels and participants
• Static-type groupchat should keep synchronicity with ‘group(room?)’ feature on our service
![Page 108: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/108.jpg)
Video/Voice Chat
![Page 109: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/109.jpg)
Problem?
• Isn’t Identity Association enough?
• AccountManage (android)
• FaceTime (iOS)
![Page 110: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/110.jpg)
VoIP
Signaling Server
Peer A Peer B
Signaling Channel Signaling Channel
RTP
Data Channel(Audio, Video)
![Page 111: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/111.jpg)
VoIP
Signaling Server
Peer A Peer B
RTP
NATs NATs
Cant’ get PeerB’s Transport Address
Cant’ get PeerA’s Transport Address
![Page 112: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/112.jpg)
NAT Traversal
![Page 113: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/113.jpg)
Hole Punching
Local Host
NATs
1. Local host connects to an external host
![Page 114: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/114.jpg)
Hole Punching
Local Host
NATs
1. Local host connects to an external host
2. NAT allocates a transport address to localhost
192.168.0.1:10001
10.100.100.100:10002
MAPPING
![Page 115: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/115.jpg)
Hole Punching
Local Host
NATs
1. Local host connects to an external host
2. NAT allocates a transport address to localhost
192.168.0.1:10001
10.100.100.100:10002
MAPPING
![Page 116: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/116.jpg)
Hole Punching
Local Host
NATs
1. Local host connects to an external host
2. NAT allocates a transport address to localhost
3. An external host can send packet to local host through mapped global address
But every NAT doesn’t work as expected
![Page 117: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/117.jpg)
STUN
STUN Server
STUN Client
NATs
Server Reflexive Transport Address
Host Transport Address
![Page 118: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/118.jpg)
STUN
Signaling Server
Peer A Peer B
NATs NATs
STUN Server
STUN Server
Do Hole Punching And
Obtain Reflexive Address
![Page 119: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/119.jpg)
STUN
Signaling Server
Peer A Peer B
NATs NATs
STUN Server
STUN Server
Exchange Obtained Reflexive Address
Through Signaling Server
![Page 120: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/120.jpg)
STUN
Signaling Server
Peer A Peer B
NATs NATs
STUN Server
STUN Server
Direct Communication Path
![Page 121: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/121.jpg)
TURN
Signaling Server
Peer A Peer B
NATs NATsTURN Server
relay packet
![Page 122: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/122.jpg)
TURN
TURN Server
TURN Client
NATs
Remote Peer
Relayed Transport Address
Server Reflexive Transport Address
Host Transport Address
NATs
![Page 123: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/123.jpg)
TURN
TURN Server
TURN Client
NATs
Remote Peer
Relayed Transport Address
Server Reflexive Transport Address
NATs
Allocation
Client ReflexiveTransport Relayed Transport
XXX.XXX.XXX.XXX:10001 YYY.YYY.YYY.YYY:10001
XXX.XXX.XXX.XXX:10001 YYY.YYY.YYY.YYY:10002
XXX.XXX.XXX.XXX:10002 YYY.YYY.YYY.YYY:10003
![Page 124: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/124.jpg)
TURN
TURN Server
TURN Client
NATs
Remote Peer A
Relayed Transport Address
NATs
Remote Peer B
NATs
Remote Peer C
NATs
![Page 125: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/125.jpg)
TURN
TURN Server
TURN Client
NATs
Remote Peer
Channel Binding
Client ReflexiveTransport Number Relayed Transport
XXX.XXX.XXX.XXX:10001 0x4001 YYY.YYY.YYY.YYY:10001
XXX.XXX.XXX.XXX:10001 0x4002 YYY.YYY.YYY.YYY:10002
XXX.XXX.XXX.XXX:10002 0x5001 YYY.YYY.YYY.YYY:10003
NATs
TODO: need peer reflexive address?
![Page 126: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/126.jpg)
XMPP Jingle
• Signaling on XMPP
![Page 127: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/127.jpg)
Serverless Messaging
![Page 128: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/128.jpg)
Protocols
• UPnP
• Bonjour(Rendezvous)
![Page 129: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/129.jpg)
Bonjour
• mDNS
• DNS-SD
![Page 130: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/130.jpg)
Find Friends
• search service by DNS PTR records
• search host for indicated service by SRV records
• search IP address by A record
• (TXT records for capability and status)
![Page 131: XMPP/Jingle(VoIP)/Perl Ocean 2012/03](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b48adf4a7959d6068b4578/html5/thumbnails/131.jpg)
TODO
• backward compatibilityの確保(iChat, etc)
• 現在の要件への現実的な対応(HTTPBinding)
• 将来の準備(WebRTC)