gevent + protobuf = abu.rpcgevent + protobuf = abu.rpc基于协程的基于协程的 RPCRPC 实现实现

赖勇浩http://laiyonghao.com

http://techparty.org

2011.5.14

升级版

RPCRPC

RPC

• Remote Procedure Call

• Application level protocol

• TCP/UDP

• HTTP

abu.rpcabu.rpc

特性• 更小（得益于 google protobuf ）• 更快（得益于 libevent ）• 同步 API （利益于 greenlet ）• 并行管线• 双向调用• TCP only

google protobufgoogle protobuf

Jeff Dean, Google fellow

• Protocol description language is a must.

• http://outerthought.org/blog/410-ot.html

google protobuf

• http://code.google.com/p/protobuf/

• 协议描述语言• C++ 、 java 、 Python and more

• service interface

Quick Example

message Person {

required int32 id = 1;

required string name = 2;

optional string email = 3;

}

Quick Example

person = Person()

person.id = 10

person.name = 'laiyonghao'

person.email = 'mail@laiyonghao.com'

with open('person.data', 'wb') as f:

f.write(person.SerializeToString())

Quick Example

service SearchService {

rpc Search (SearchRequest) returns (SearchResponse);

}

geventgevent

greenlet + libeventgreenlet + libevent

greenlet

• green thread– user space– pseudo-concurrently– scheduled by VM– http://en.wikipedia.org/wiki/Green_threads

• Python 的 green thread

PEP 342

def echo_handler(sock): while True: try: data = yield nonblocking_read(sock) yield nonblocking_write(sock, data) except ConnectionLost: pass # exit normally if connection lost

libevent

• 提供指定文件描述符事件发生时调用回调函数的机制

• timeouts, signals

ExamplesExamples

echoecho

echo.proto

• message Packet{• required string text = 1;• }

• service EchoService{• rpc echo(Packet)returns(Packet);• }

s_echo.py

• @abu.rpc.check_service• class EchoService(echo.EchoService):• @abu.rpc.send_return• def echo(self, controller, request, done):• return request

• service = EchoService()• services = (service,)• server = abu.rpc.Server(('0.0.0.0', 10086), services)• print 'serving...'• server.serve_forever()

c_echo.py

• channel = abu.rpc.Channel(('127.0.0.1', 10086), ())• server = abu.rpc.Proxy(echo.EchoService_Stub(chann

el))

• for i in xrange(10):• req = echo.Packet()• req.text = 'hello'• print 'send:%s' % req.text• resp = server.EchoService.echo(req)• print 'recv:%s' % resp.text

chat roomchat room

chat.proto• message HelloReq{• required string name = 1;• }

• message HelloResp{• required string name = 1;• required bool succ = 2;• optional string info = 3;• }

• message ByeReq{• required string name = 1;• }

• message ByeResp{• required string name = 1;• required bool succ = 2;• optional string info = 3;• }

• message SayReq{• required string name = 1;• required string text = 2;• }

• message SayResp{• required string name = 1;• required bool succ = 2;• optional string info = 3;• }

• message Message{• required string name =

1;• required string text = 2;• }

• service ChatClientService{• rpc say(Message)retur

ns(abu.rpc.Void);• }

chat.proto

• service ChatServerService{• rpc hello(HelloReq)ret

urns(HelloResp);• rpc bye(ByeReq)return

s(ByeResp);• rpc say(SayReq)return

s(SayResp);• }

s_chat.py:ChatServerService.hello

• @abu.rpc.send_return• def hello(self, controller, request, done):• name = request.name• resp = chat.HelloResp()• resp.name = name• if name in self._clients:• resp.succ = False• resp.info = u' 用户名（ %s ）已经有人使用了。 ' %

name• return resp• self._clients[name] = Client(controller.socket)• resp.succ = True• return resp

s_chat.py:ChatServerService.say• @abu.rpc.send_return• def say(self, controller, request, done):• name = request.name• resp = chat.SayResp()• resp.name = name• if name not in self._clients:• resp.succ = False• resp.info = u' 无效的用户名（ %s ） ' % name• return resp• msg = chat.Message()• msg.name = name• msg.text = request.text• for n, client in self._clients.iteritems():• client.proxy.ChatClientService.say(msg)• resp.succ = True• return resp

c_chat.py:ChatClientService.say

• @abu.rpc.check_service• class ChatClientService(chat.ChatClientService):• @abu.rpc.send_return• def say(self, controller, request, done):• print u'%s 说： %s' % (request.name, re

quest.text)• return

Thx~Thx~

twitter.com/laiyonghaoweibo.com/gzlaiyonghao