DeNA loves PerlTokiharu Noto

(DeNA Co., Ltd.)

1

agenda

• How much / why does DeNA love Perl?

• outline of Perl usage in DeNA

• YAPC-driven Open Source from DeNA

• mobamail: mail delivery for mobile phone

• Mobage-town Open Platform

• ambivalent love for Perl

2

DeNA services powered by Perl

2006

2007 2008

2004

2006

2008

2004

一部機能2009

3

How many DeNA engineers love Perl?

• about 100 engineers

• > 50 engineers in mbga.jp team

• YAPC::Asia Tokyo 2009 speakers

• Hideo Kimura (modern Catalyst)

• Toru Yamaguchi (Perl DB test using MySQL::Sandbox)

• Tadashi Matsuda (MobaSiF::Template)

4

Why does DeNA love Perl?

• bidders: Java + Oracle

• In 2003, Kawasaki has started developing mbok.jp with Perl

• In 2006, ex-bidders engineers joined to Moba* services

• dev/edit/exec speed++, readability++, productivity++ (also for biz people)

• The whole DeNA have loved Perl since

5

DeNA scalable web architecture

LoadBalancer

LoadBalancer

httpd+fcgiservers

M

S S S ...

M

S S S ... MySQL+Senna

MySQLw/replication

...

... partitioning

...

memcached

mbga > 90

mbga > 1300servers

LVSDNS RR

6

MobaSiFDeNA original WAF

• Open Source since YAPC::Asia Tokyo 2008

• pros

• common modules for mobile web

• simple, light-weight, fast

• stable

• cons

• no PSGI/Plack ;-)

• $_::F, $rhData

• no test...

• too free for a large team...

Apache

mod_fastcgi

MobaSiF

pictgraph converter

dispatcher

handset ditector

template engine

web applications

7

YAPC-drivenOpen Source

• 2008

• MobaSiF

• 2009

• MobaSiF::Template (Matsuda)

• mobamail - fast mail delivery daemon for mobile phones

8

mail deliveryfor mobile phones

• Mobile phone carriers deny SMTP connection at their own discretion (裁量).

• “spam?”

• especially when senders try to send mails with high speed...

• conflict with our wish “more speeeeedy!”

9

mail delivery for mobile phones

• 1.09 億 (109 million) users / 3 carrier ≒ ISP

• MX (cf. dig -t MX <domainname>)

• docomo.ne.jp: 4 IP addresses

• ezweb.ne.jp: 1 IP address

• softbank.ne.jp: 1 IP address

10

!!"#$!%&'(&)!

!!!!

"#*!%&'(&)!

+,PC -./0,"#$%&'(&)123456-78"#*%&'(&)9:1;<=>?@A4BC1DEFG!

!!

"#$!%&'(&)!

"#*!%&'(&)!

H,IJ-./0,"#$%&'(&)-31KLM6-78"#*%&'(&)9:1;<=>?@A4BC1D4NG!

!!

OPQR,A! OPQR,B OPQR,C

mail deliveryfor mobile phones

11

before mobamail

• At first (2004-), we used qmail, postfix

• slow injection

• main: sendmail command

• SMTP/QMTP...

• We have to make our own original server...

12

mobamail

• implemented in pure Perl

• ≠ MTA (no SMTP receive, relay)

• injection = insert into a table in MySQL

• high speed table2mail to mobile phones

• max speed (depend on carrier-side config)

• docomo+au: 500,000 mails/h

softbank: 20,000 mails/h

13

mobamail strategy• provide tuning params

for best effort under the carriers permission:

• num of connection

• connect() interval

• mail per connection

• mail interval (in one connection)

• interval after denied

• connection closed, timeout, ...

• timeout

• connect()

• wait for response

• unused connection

• error count

=> retry_interval

14

mobamail architecture

‘dist’ daemon

MySQL

‘docomo’

‘au’

‘softbank’

‘default’‘send’ daemon

processes

MX

MXdocomo...

MX

MXau...

MX

MXsoftbank...

MX

MXother ISP...

IO::Select

one server

SMTP

15

current operationMX

MXdocomo...

MX

MXau...

MX

MXsoftbank...

MX

MXother ISP...

SIELLA ENGINE

mobamail

mobamail

better performance

without tuning16

mobamailwill be available at...

• http://sourceforge.jp/projects/moba/

• http://d.hatena.ne.jp/tokiharu/ (w/RSS)

17

Mobage-town Open Platform18

DeNA & Perl, again

19

DeNA dev policy

• すべてのエンジニアがビジネスの成功にコミットする

• 設立から数年赤字で苦労した...

• 「たくさんの人に使ってもらいたい」というエンジニアの原点

• お金で解決せず技術で =>「安く・速く」

• ビジネス/数字を意識しているから企画にも関われる

• ただ、きれいな技術が二の次になるという面も否めない...

20

ambivalent love• 100 人近く Perl エンジニアがいるのに CPAN コミットがない。なぜ?

• 他社特許侵害のリスク (担保する仕組みは欲しい)

• やりたい人がいたら推奨しようとしているが...

• DeNA の Perl での開発方針

• 「一筆書き文化」: 再利用性や汎用性の設計に時間をかけるくらいなら、そういったことは考えずに書く方が速い (おおむね)

• サービス単位の独自性・機動性重視 = モジュールの共有なし

→ 割り切りの思想、逆転の思想、これで勝ってきた面も。

• ただ CPAN の文化 (組織をまたがったコードの共有) と相容れない!

21

DeNA gonna love CPAN

• 正直、あんまり自覚がなかった• Some pilot projects depend on a lot of CPAN

modules (and Catalyst)

• 今までの強みを活かした折衷案を• dev policy for the large team

• more test code

• MobaSiF next generation

22

DeNA loves Perl!

• a Perl community cosists of 100 engineers

• Let’s write code together!

• as DeNA software engineer / architect

• as developer of Mobage-town applications

• as CPAN author

23

DeNA loves Perl!

24