Upload
kuchinskaya
View
454
Download
1
Embed Size (px)
Citation preview
Современная разработка в Mail.ru
На Perl и не только
Myth
es Perl 5.6 Apache 1
CGI
?
How many servers enough to process
100k requests per second
with 0.01s delay?
The q
uestio
n
Users
Read
An
d w
ant
An
d w
ant avatars
From
everywh
ere
Man
y
4Krps
An
d fast
4Krps
0.01s
Filin
Intro
du
ce: Filin
4Krps
Сервер на C, который отдает
аватарки и иконки
Wh
at the so
urce
?
Ow
n…
Mail, Fb
, Ok, G
ravatar, ICQ
, …
… … …
Filin
Ow
n d
ata on
disk
4Krps
disk
1K
rps
Filin
Wh
ere the o
thers?
4Krps 3Krps
disk
1K
rps
?
Filin
Co
den
ame “A
vatard”
4Krps 3Krps
Сервис, умеющий искать и отдавать аватарки. Быстро
disk
1K
rps
IProto
Filin
HT
TP – O
verhead
. IPro
to!
4Krps 3Krps
disk
1K
rps
struct packet { uint32_t type, uint32_t length, uint32_t sequence, char[] data }
Srv
IProto
Filin
Avatard
insid
e: the server
4Krps 3Krps
Srv
Srv
IProto
Filin
Avatard
: servers
4Krps 3Krps
DB Srv
Srv
IProto
Filin
Take data fro
m d
atabase
4Krps 3Krps
M Srv
Srv
IProto
Filin
Rep
licated
4Krps 3Krps S
M1 Srv
Srv
S1
Mn Sn
IProto
Filin
Of co
urse, sh
arded
4Krps 3Krps
Tarantool
M1 Srv
Srv
Que
S1
Mn Sn
IProto
Filin
Bu
t wh
ere they fro
m?
4Krps 3Krps
Tarantool
очередь
M1 Srv
Srv
Que
S1
Mn Sn
IProto
Filin
From
the in
ternet )
4Krps
…
3Krps
Res
Tarantool
M1 Srv
Srv
Que
S1
Mn Sn
IProto
HTTP
Filin
Svc
Som
e HT
TP R
equ
ests
4Krps
…
3Krps
3Krps
Res
Tarantool
M1 Srv
Srv
Que
S1
Mn Sn
IProto
HTTP
Filin
Svc
Go
t a pro
blem
!
4Krps
…
3Krps
3Krps
Res
Tarantool
0.01s
3s
1.5s
2.5s
Ho
w em
ail wo
rks?
письмо SMTP Server
SMTP Mail.ru
Хранилище
http://e.mail.ru
Агент Mail.ru
Письмо или список писем
Когда узнаем
Когда показываем
+ аватарки
M1 Srv
Srv
Que
S1
Mn Sn
swp
IProto
IProto
HTTP
Filin
Svc
MTA
Imp
rove!
4Krps
5Krps
disk …
3Krps
3Krps
5Krps Res
Tarantool
>>3s
M1 Srv
Srv
Que
S1
Mn Sn
IProto
IProto
HTTP
Filin
Svc
MTA
Secon
d p
rob
lem: R
AM
4Krps
5Krps
…
3Krps
3Krps
5Krps Res
Tarantool
M1 Srv
Srv
Que
S1
Mn Sn
swp
IProto
IProto
HTTP
Filin
Svc
MTA
RA
M is n
ot ru
bb
er. Use d
isk
4Krps
5Krps
disk …
3Krps
3Krps
5Krps Res
Tarantool
Зачем хранить в памяти неиспользуемые?
M1 Srv
Srv
Que
S1
Mn Sn
swp
IProto
IProto
HTTP
Filin
Svc
MTA
Well d
on
e!
4Krps
5Krps
disk …
3Krps
3Krps
5Krps Res
Tarantool
Language?
C?
Avo
id it
Perl! TIMTOWTDI
Fast dev early deploy
M1 Srv
Srv
Que
S1
Mn Sn
swp
IProto
IProto
HTTP
Mo
dern
langu
age!
Res
Perl 7!
Well kn
ow
n Perl 5
.
Perl 6. O
’RLY?
Perl 7! N
ow
!
M1 Srv
Srv
Que
S1
Mn Sn
swp
IProto
IProto
HTTP
Mo
dern
langu
age!
Res
Perl 7! (5.16)
M1 Srv
Srv
Que
S1
Mn Sn
swp
IProto
IProto
HTTP
Mo
dern
async fram
efork!
Res
Perl 7! AnyEvent
M1 Srv
Srv
Que
S1
Mn Sn
swp
IProto
IProto
HTTP
Mo
dern
async fram
efork!
Res
Perl 7! AnyEvent IProto::Server
M1 Srv
Srv
Que
S1
Mn Sn
swp
IProto
IProto
HTTP
Mo
dern
async fram
efork!
Res
Perl 7! AnyEvent IProto::Server (40k rps/core)
M1 Srv
Srv
Que
S1
Mn Sn
swp
IProto
IProto
HTTP
Mo
dern
async fram
efork!
Res
Perl 7! AnyEvent IProto::Server (40k rps/core) HTTP::Server
M1 Srv
Srv
Que
S1
Mn Sn
swp
IProto
IProto
HTTP
Mo
dern
async fram
efork!
Res
Perl 7! AnyEvent IProto::Server (40k rps/core) HTTP::Server (8k rps/core)
M1 Srv
Srv
Que
S1
Mn Sn
swp
IProto
IProto
HTTP
Mo
dern
async fram
efork!
Res
Perl 7! AnyEvent IProto::Server (40k rps/core) HTTP::Server (8k rps/core) Tarantool
M1 Srv
Srv
Que
S1
Mn Sn
swp
IProto
IProto
HTTP
Mo
dern
async fram
efork!
Res
Perl 7! AnyEvent IProto::Server (40k rps/core) HTTP::Server (8k rps/core) Tarantool HTTP
M1 Srv
Srv
Que
S1
Mn Sn
swp
IProto
IProto
HTTP
Mo
dern
async fram
efork!
Res
Perl 7! AnyEvent IProto::Server (40k rps/core) HTTP::Server (8k rps/core) Tarantool HTTP EV
M1 Srv
Srv
Que
S1
Mn Sn
swp
IProto
IProto
HTTP
Mo
dern
async fram
efork!
Res
Perl 7! AnyEvent IProto::Server (40k rps/core) HTTP::Server (8k rps/core) Tarantool HTTP EV XS
Sou
rces availab
le
Perl 7! AnyEvent IProto::Server (40k rps/core) HTTP::Server (8k rps/core) Tarantool HTTP EV XS
github.com/Mons/AnyEvent-IProto
github.com/Mons/AnyEvent-HTTP-Server-II
Twiggy, Twisted,
Node.js,
(but not nginx ;)
Faster than
Perl 5.6? Perl 5.6!
M1 Srv
Srv
Que
S1
Mn Sn
swp
IProto
IProto
HTTP
Res
Perl 7!
Apache! Apache?
CGI! CGI?
Perl 5.1
6 vs Perl 5
.8
- Released 11.03.2013 - Lower memory usage up to 25% - Lower CPU usage up to 10% - The best Unicode 6 support - Many modern features - Latest libraries - Support for new libraries - No old bugs - New regular expression - Async frameworks - Green threads
Perl 5.16 - Released 31.01.2006 - Buggy - Obsoleted!
Perl 5.8
Database?
M1 Srv
Srv
Que
S1
Mn Sn
swp
IProto
IProto
HTTP
No
SQL D
atabase
Res
Tarantool
Tarantool - High performance - Latest stable - Built in Lua - Multiple indexes - Composite indexes
M1 Srv
Srv
Que
S1
Mn Sn
swp
IProto
IProto
HTTP
An
d also
qu
eue
Res
Tarantool
Tarantool - High performance - Latest stable - Built in Lua - Multiple indexes - Composite indexes - Lua-based queue
How many servers enough to process
100k requests per second
with 0.01s delay?
So, th
e qu
estion
…
The an
swer is…
Mo
re correct an
swer ;)
Profit: effective highload IProto: 120k rps / server
HTTP: 40k rps / server
Fast development – early deploy
Profit: effective highload
1400% reservation
Appliance High performance applications
U Complex logic with many data sources
Mons Anderson
Р у к о в о д и т ел ь о тд ел а , П о ч т а @ M a i l . r u
< m o n s @ c p a n . o r g >
< v. p e r e p e l i t s a @ c o r p . m a i l . r u >