XMPP/Jingle(VoIP)/Perl Ocean 2012/03

XMPP Based Realtime Communication

Framework Suite

!

加藤 亮 2012/03

Ocean

Realtime

• Data (micro contents)

• Delivery Method ( Push )

• UI

PUSH Friendly Contents

• Notification

• Activity Stream

• Message, Chat

• Presence

PUSH

• Long Polling

• XMPP

• HTML5 (SSE, WebSocekt)

• Smart phone (APNS, C2DM)

UI

• Ajaxify - auto view updating

Ocean

Framework Suite

• Front Server

• Cluster

• HTTP Binding

• P2P(Audio/Video chat) 予定

Protocol

• XMPP Based

• XMPPのサブセットをサポート

• XMPPと内部で互換性を持たせた数種類の独自プロトコル

• External WebSocket Binding

• HTTP Binding ( WebSocket/SSE/XHR )

Jabber/XMPP

examples

• google talk

• facebook chat

• PSN

• iMessage

• google wave ( S2S )

Stream Procedure

• Stream Initialization

• TLS Negotiation

• Authentication

• Resource Binding

• Session Establishment

• Roster Request

• Initial Presence

Stanza

• Feature

• Auth

• Message

• Presence

• IQ

ID

• JID (example: lyokato@mixi.jp/resource )

• Full JID (lyokato@mixi.jp/resource)

• Bare JID (lyokato@mixi.jp)

Authentication• SASL

• DIGEST-MD5

• PLAIN ( under TLS )

• EXTERNAL ( for S2S TLS )

• We need

• OAuth 2.0?

• Cookie/Ticket ( for WebSocket )

Message

Message

• service notification (type=headline)

• chat message (type=chat)

• group chat message (type=groupchat)

• normal ? message (type=message)

Presence

Presence

• presence

• subscription

• Ocean doesn’t support subscription

• extension

• join/leave a room

Presence

• show (chat, away, dnd, xa )

• status (custom message)

• priority

Presence traps

• protocol design

• presence:connection = 1:1

• de facto usage

• presence:user = 1:1

Presence De Facto Usage

• one presence for one user

• we don’t need priority

• ‘chat’ and ‘away’ are enough for ‘show’ param

Resource Binding

• node@domain/resource

• you can choose ‘home’ or ‘work’

• de facto usage

• users and client softwares don’t mind what the resource is.

• so server generates random string (like a cookie )

Session Establishment

• Historical Reason

• Do nothing

IQ

• Roster ( Friend List )

• vCard ( Profile )

• bind, session, ping, disco, etc...

• packets should be back to sender

Roster & vCard• like OpenSocial’s ‘People Data API’

Roster

• there isn’t profile image, which is included in vCard

• no pagination control by default

vCard traps

• just for profile image

• but base64 encoded binary

• we want URL

Subscription & Roster Push

• Friend Request

• Relationship State Control

C2S & S2S

• Client to Server

• Server to Server

• Ocean doesn’t support S2S

C2S Example

gmail

User A User B

C2S StreamC2S Stream

S2S Example

gmail.com

User A User B

chat.facebook.com

User DUser C

S2S Stream

S2S Problems

• We have to manage new database that represents ‘relationships’ between users of our service and ones of other service.

• We have to manage a white list or spam control. ( like OpenID )

• Authentication Compatibility Cost

• Scaling Cost

• Error Handling

Implementation

Other Implementations• I/O Multiplexing

• Node.js

• Apache MINA, naggati

• Jabber/XMPP software

• ejebberd

• DJabberd

• Openfire

Ocean’s XMPP support

• subset

• customized

• fixed features(not pluggable )

• assumed cluster installation

Why subset?

• Ocean is just for Web-based Social Network Service

• Create/Modify friendships on the web site.

• S2S costs too much

• Developers have to write only minimum number of event handlers.

Protocol Customized

• resource binding & roster/vCard responses support extra-attributes.

• Because of regardless of HTTP federation, especially URL

• WebSocket binding (experimental )

• try demo application

• Simple-MUC

Not flexible and pluggable?

• It may cause complication. requires detailed protocol knowledge, framework knowledge.

• When the servers are distributed, front-server-plugins require complicated spec. See ‘Cluster’ section.

• There are de-facto-standard specs.

Why full scratch?

• Proprietary Reason

• We may need non-standard features

• Rapid Updating (community based development requires many arguments)

• Architectural Dependencies

• No Money

Inside Ocean

Getting Started

• All you need to do is ...

• generate project template

• edit configuration file

• prepare your handler class

Generate Project Template

• ./ocean

• cd Foobar

Configuration

• vi conf/ocean.yml

Write Handler Class

• inherit Ocean::Handler

• implement all event handler methods

Ocean Components

Server Stream Manager

Stream Stream Stream

Handler

Daemonizer Signal Handler Listener Stream Builder

Stream Components

IO Components

Inside Stream

Stream Manager

Encoder

Decoder

Socket

Protocol States

Client

SASL

Active

...

ServerHandler

Events

Problem?

• Bad performance

• too many call stack on each event

• each component consume memory

Benefit

• Testability

• You can switch to stub components

• Customizability

• You can switch Encoder/Decoder

Pay attention to memory usage

• 4byte on a Stream object

• 4x10,000byte on a Server witch has 10,000 connection

• bless Array instead of Hash

• use Config instance as singleton object

• use delegate object for event handling

Your Service

Databases

Handler

Handler

Users RelationshipsConnection Info

User A Home chat

User A Work away

User B Home chat

User C Home chat

Files

Profile Pictures

Handler uses 2 or 3 kinds of store

• Persistent Data Store (users, relationships)

• Half Persistent Data Store. Same lifecycle as XMPP service (connection map)

• Cache system ( if needed )

Handler Implementation

Handler Example

• modules under Ocean::Standalone::Handler namespace are good example

Server Events

• on_node_init

• on_node_exit

Stream Events

• on_sasl_auth_request

Stream Events

• on_bind_request

Stream Events

• on_message

Stream Events

• on_presence

• on_initial_presence

• on_unavailable_presence

Presence doesn’t scale• user_001 login-ed ( send initial-presence )

• get all friends of user_001

• get all available-resources of the friends

• send user_001’s presence to all the available-resources.

• send presences of the all available-resources to user_001

It’s important to limit the maximum number of friends.

Stream Events

• on_silent_disconnection

Stream Events

• on_oster_request

Stream Events

• on_vcard_request

Stream Events

• on_too_many_auth_attempt

Start & Stop

• bin/ocean-start

• bin/ocean-start --daemonize

• bin/ocean-stop ( if daemonized )

Cluster

• C10K Problem on Persistent Connection

• Blocking IO multiplexing

• Message delivery/Presence broadcast

Message Delivery

XMPP Server 1

User A (Home)

User B (Home)

XMPP Server 2

User C (Home)

Delivery Server

Connection Map

User A Home 1

User A Work 2

User B Home 1

User C Home 2

User A (Work)

Use 2 kinds of gearman

• Background Job Dispatcher

• Inbox ( 1 inbox for each front-server )

Message Delivery

XMPP Server 1 XMPP Server 2

Delivery Server

Connection Map

User A Home 1

User A Work 2

User B Home 1

User C Home 2

Background Job Dispatcher (gearmand)

inbox1 (gearmand)

inbox2 (gearmand)

Message Delivery

XMPP Server 1 XMPP Server 2

Delivery Server

Connection Map

User A Home 1

User A Work 2

User B Home 1

User C Home 2

Background Job Dispatcher (gearmand)

inbox1 (gearmand)

inbox2 (gearmand)

Message Delivery

XMPP Server 1 XMPP Server 2

Delivery Server

Connection Map

User A Home 1

User A Work 2

User B Home 1

User C Home 2

Background Job Dispatcher (gearmand)

inbox1 (gearmand)

inbox2 (gearmand)

Connection Map• too many ‘read’

• ‘write’ on resource-binding, presence-change

• Implementation

• TokyoTyrant

• HandlerSocket

• must not be volatile

Ocean::Cluster

• Ocean::Cluster::Gearman::Server

• Ocean::Cluster::Gearman::Worker

Cluster support on front server

• set config ‘handler_class=Ocean::Cluster::Gearman::Server::Handler’

• set inbox and background-job-dispatcher

Front server components

XMPP Server

inbox (gearmand)

Background Job Dispatcher (gearmand)

Ocean::Cluster::Gearman::Server::Handler

Build worker project

!

• same as front server

• build project-template

• edit configuration

• write your handler class

Generate Project Template

• ocean_cluster_gen --project=FooWorker --handler=FooWorker::Handler

• cd FooWorker

Configuration

• set information about all your inboxes

• set information about background-job-dispatcher

Worker

Worker side (delivery server) components

inbox0 (gearmand)

Background Job Dispatcher (gearmand)

inbox1 (gearmand)

Handler

Serializer

Subscriber Deliverer

Process Manager

Connection Map

User A Home 1

User A Work 2

User B Home 1

User C Home 2

DB (users, relationships, etc...)

Transaction

• 同じユーザーからのイベント順序が壊れないように調整が必要

Operation

Balancing

• Round-Robin

• DNS SRV Record

• _xmpp_client._tcp.chat.mixi.jp 86400 IN SRV 10 20 5222 chat1.mixi.jp

• _xmpp_client._tcp.chat.mixi.jp 86400 IN SRV 10 5 5222 chat2.mixi.jp

chat1.mixi.jp

User A (Home)

Service Status Store

name connections wait

chat1 100000 2

chat2 1000 1

chat3 100000 2

chat2.mixi.jp chat3.mixi.jp

Update Periodically

Status API

Service Status API and Client Hack

chat1.mixi.jp

User A (Home)

Service Status Store

name connections wait

chat1 100000 2

chat2 1000 1

chat3 100000 2

chat2.mixi.jp chat3.mixi.jp

Update Periodically

Status API

Service Status API and Client Hack

chat1.mixi.jp

User A (Home)

Service Status Store

name connections wait

chat1 100000 2

chat2 1000 1

chat3 100000 2

chat2.mixi.jp chat3.mixi.jp

Update Periodically

Status API

Service Status API and Client Hack

Update & Reboot

• Worker side handler includes service-code, so it’s required updating and rebooting frequently. Worker’s job is stateless. So, it’s no problem.

• Front-end server doesn’t include service-code. So you don’t need to reboot it frequently, but when you want to do it, you have to pay attention.

Shutdown• You can’t shutdown server without client

awareness.

• You have to write re-connection on client side code

• Problem: algorithm distribution case

• At graceful shutdown, unavailable presences are handled correctly

• Problem: at non-graceful shutdown

Front-server Broken

• フロントがぶっ壊れたときのために、ConnectionMapから、壊れたサーバーにつないでたユーザー全員のunavailable

presenceを処理するバッチを用意する必要がある

Front-server maintainance

• algorithm分散のフロントを定期再起動するときは、代替が先に必要

• group_id = get_group(user_id)

• host = get_host(group_id)

• 代替サーバーを準備し、group/host

マップを書き換えてから、shutdown

Service(Client-side) Strategy

Device/Platform

• PC Web ( HTTP Binding )

• PC Desktop App(AIR, Titanium, or Native)

• Smartphone

• Tablet

• Browser extension

Service

• 1to1 Messaging

• Room? (group chat)

• Voice/Video chat

• Push Notification

Ocean supports

• WebSocket hybi-draft-10 support

• HTTPBinding(WebSocket/SSE/XHR)

HTTP Binding

Methods

• Polling ( Ajax ) + Post API (+APNS|C2DM)

• Long Polling ( Ajax, Comet ) + Post API

• ServerSentEvent + Post API

• WebSocket

• TCP XMPP

‘Stateless’ problems

• Different way of connection management from ordinary persistent connection

• single user-session may open many tabs

• user moves in short time. too many re-connection

• how to detect user to leave? (leave or move)

server 1

User A

server 2 server 3

Connection Map

User A Home 1

User A Work 2

User B Home 1

User C Home 2

too many ‘write’ and ‘presence’ in short time

open many tabs or hop http-link in short time

TabTab

Tab

mod_proxy(_balancer)

User B

when user leaves?

Solution 1 Algorithm Based Distribution

• ユーザー毎の宛先を固定することによって、Connection Mapを利用しなくても、特定のユーザーにパケットを届けることは可能になる

• ユーザーがWebサービスを開いてるかどうかは関係なくなる

Problem

• Connection Mapを利用しないと、正確なプレゼンスを把握できなくなる

Solution 2. Stream Aggregation

• 同一cookieを持つ接続をまとめあげるSessionオブジェクトを利用する

• 状態管理はSessionオブジェクトで工夫

• Pending TimerでLeaveの検知

User B

hb1.mixi.jp hb2.mixi.jp hb3.mixi.jp

User A

TabTab

TabUser C

TabTab

Tab

Connection Map

User A Home 1

User A Work 2

User B Home 1

User C Home 2Aggregate

connections as single session

algorithm based distribution: server_node=algorithm(user_id)

Manager Manager Manager

use timer to detect users to leave.

(*) number of simultaneous Ajax request directed to same domain is limited, so we need many nickname for each hb servers. (if we adopt long polling)

until the expiration, server should buffer messages if received

Server Stream Manager

User A Stream

User A Stream

Handler

Session

Handle Multiple Streams as One Single Session

User B Stream

User B Stream

Session

Auth

ResourceBinding

SessionEstablishment

InitialPresence

Handshake

Tasks

• We need to

• prepare social ad system

• improve UI (moveless)

PubSub Support

• XEP-0061 Publish Subscribe

• We don’t need full-spec support

• don’t allow use to publish/subscribe

• server publish event

Publish Event from Web Service

XMPP Server 1

User A (Home)

User B (Home)

XMPP Server 2

User C (Home)

Delivery Server

User A (Work)

Web ServiceUserEvents

(Notification, Activity)

XMPP (HTTPBinding) Streams

Web Service

UserEvents (Notification,

Activity)

APNSC2DM EmailWebHook PubsubHubbub

DeliveryCondition: ContentKind, UserPresence

Groupchat

Overview

• XMPP has MUC extension spec. but it’s over spec, so we need arrangement

• Costs too much like presence

• limit numbers of channels and participants

• Static-type groupchat should keep synchronicity with ‘group(room?)’ feature on our service

Video/Voice Chat

Problem?

• Isn’t Identity Association enough?

• AccountManage (android)

• FaceTime (iOS)

VoIP

Signaling Server

Peer A Peer B

Signaling Channel Signaling Channel

RTP

Data Channel(Audio, Video)

VoIP

Signaling Server

Peer A Peer B

RTP

NATs NATs

Cant’ get PeerB’s Transport Address

Cant’ get PeerA’s Transport Address

NAT Traversal

Hole Punching

Local Host

NATs

1. Local host connects to an external host

Hole Punching

Local Host

NATs

1. Local host connects to an external host

2. NAT allocates a transport address to localhost

192.168.0.1:10001

10.100.100.100:10002

MAPPING

Hole Punching

Local Host

NATs

1. Local host connects to an external host

2. NAT allocates a transport address to localhost

192.168.0.1:10001

10.100.100.100:10002

MAPPING

Hole Punching

Local Host

NATs

1. Local host connects to an external host

2. NAT allocates a transport address to localhost

3. An external host can send packet to local host through mapped global address

But every NAT doesn’t work as expected

STUN

STUN Server

STUN Client

NATs

Server Reflexive Transport Address

Host Transport Address

STUN

Signaling Server

Peer A Peer B

NATs NATs

STUN Server

STUN Server

Do Hole Punching And

Obtain Reflexive Address

STUN

Signaling Server

Peer A Peer B

NATs NATs

STUN Server

STUN Server

Exchange Obtained Reflexive Address

Through Signaling Server

STUN

Signaling Server

Peer A Peer B

NATs NATs

STUN Server

STUN Server

Direct Communication Path

TURN

Signaling Server

Peer A Peer B

NATs NATsTURN Server

relay packet

TURN

TURN Server

TURN Client

NATs

Remote Peer

Relayed Transport Address

Server Reflexive Transport Address

Host Transport Address

NATs

TURN

TURN Server

TURN Client

NATs

Remote Peer

Relayed Transport Address

Server Reflexive Transport Address

NATs

Allocation

Client ReflexiveTransport Relayed Transport

XXX.XXX.XXX.XXX:10001 YYY.YYY.YYY.YYY:10001

XXX.XXX.XXX.XXX:10001 YYY.YYY.YYY.YYY:10002

XXX.XXX.XXX.XXX:10002 YYY.YYY.YYY.YYY:10003

TURN

TURN Server

TURN Client

NATs

Remote Peer A

Relayed Transport Address

NATs

Remote Peer B

NATs

Remote Peer C

NATs

TURN

TURN Server

TURN Client

NATs

Remote Peer

Channel Binding

Client ReflexiveTransport Number Relayed Transport

XXX.XXX.XXX.XXX:10001 0x4001 YYY.YYY.YYY.YYY:10001

XXX.XXX.XXX.XXX:10001 0x4002 YYY.YYY.YYY.YYY:10002

XXX.XXX.XXX.XXX:10002 0x5001 YYY.YYY.YYY.YYY:10003

NATs

TODO: need peer reflexive address?

XMPP Jingle

• Signaling on XMPP

Serverless Messaging

Protocols

• UPnP

• Bonjour(Rendezvous)

Bonjour

• mDNS

• DNS-SD

Find Friends

• search service by DNS PTR records

• search host for indicated service by SRV records

• search IP address by A record

• (TXT records for capability and status)

TODO

• backward compatibilityの確保(iChat, etc)

• 現在の要件への現実的な対応(HTTPBinding)

• 将来の準備(WebRTC)