Andrey Mysqlnd

8/13/2019 Andrey Mysqlnd

1/42

1Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database

ysqlndA new raPHP Vikinger

Skien, NorwayJune, 9 th200

Andrey Hristo! "andrey# ysql$%o &


2/42


'ho is talking to you(

Andrey Hristov works for MyS! as a Software Developer MyS! "onnectors departe#ent author of MyS! $vent Scheduler

%#a&ister e'a#( hacks PHP since a&es has developed #ost of #ys)lnd takes care of e't*#ys)li to&ether with +eor& ,ichter

--- tries to answer your )uestions . /ust ask0 --- replies to %so#e( #ails send to andrey1#ys)l-co#


3/42


'hat does )mysqlnd* mean(

MySQL Server

The world's most populr ope! sour"e dt#se 2LAM$2 %!inu'3 Apache3 MySQL3 $%$* Perl * Python(

http4**dev-#ys)l-co#*

mys&l!d MySQL !tive driver (or $%$

)tive driver to "o!!e"t (rom $%$ to the MySQL Server Ti&htly inte&rated into PHP3 able to support PHP &oodies )ot !ew progrmmi!g A$* http4**dev-#ys)l-co#*downloads*connector*php5#ys)lnd*


4/42

+Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database

'hat is mysqlnd(

!ibrary and not a new e'tension not e'posed to the userland i#ple#ented in "

Drop5in replace#ent for lib#ys)l works with e'istin& software al#ost4 so#e e'peri#ental functions are no lon&er available

"o#pile5ti#e switch for e't*#ys)li e't*#ys)l support in the develop#ent tree PDO*MyS! support planned

PHP version independent4 PHP 63 PHP 7 theoretically even PHP 83 you #i&ht be able to port it0


5/42

,Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database

lib#ys)l#ys)lnd

MyS! native driver for PHP 5 #ys)lnd

How PHP %onne%ts to +yS-

9end $n&ine

e't*#ys)l PDO*MyS!e't*#ys)li

MyS! Server

MyS! "lient !ibrary 5 lib#ys)l

: MyS! "lient !ibrary option re#ains3 lib#ys)l is not deprecated

: ;ou can


6/42

-Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database

'hy mysqlnd( .he /ros 01

+P!*Dual5!icensehttp4**www-&nu-or&*copyleft*&pl-ht#l

http4**www-#ys)l-co#*co#pany*le&al*licensin&*

f you want closedsource and earn #oney3 share your win and pay

for a co##ercial license=

?SD5Stylehttp4**www-php-net*license


7/427Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database

'hy mysqlnd( .he /ros 02

!ib#ys)l co#es with the MyS! Server updatin& lib#ys)l #eans updatin& the MyS! Server bound to the server release cycle

#ys)lnd is a library %stored in the e't*#ys)li directory( up&radin&3 e-&- after a bu& fi' is si#ple has its own release and develop#ent cycle no lon&er linkin&*co#pilin& issues no need to install lib#ys)l on the build server

leaner code

Ti&htly coupled with PHP and 9end $n&ine can do #a&ic to &et #ore /uice out of your "P@%s( better inte&ration into PHP %see below(


8/42.Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database

'hat are the %ons(

!ess tested than e't*#ys)li 5 lib#ys)l co#bo we have #ore than doubled the nu#ber of tests

and pass the# all we have even found lib#ys)l bu&s with the new tests we have done #ainly "!> tests no web tests we have no*few user feedback

So#e %e'peri#ental( functions are no lon&er available mysqli_embedded_*

,eplication related3 e-&- mysqli_enable_rpl_parse

+ood old ti#es are &one---


9/42/Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database

A3ra%ada3ra4 memory limit

Syste# Me#ory

9end $n&inephp-ini4 #e#oryBli#it C EFM

EF

e't*#ys)li #ys)lnd

e#alloc%(

#alloc%(

lib#ys)l

Me#ory li#it now supported0



A3re%ada3ra4 /ersistent %onne%tions

Apache

Worker

"onnection "ache

Worker E

"onnection "ache

,e)uest ,e)uest ,e)uest

Persistent connections like e't*#ys)l and lib#ys)l saves connection overhead4 G6 for Dell DID Store works with Apache prefork*threads and ast5"+> Per process resp- per thread cache

uture4 optional fla& to reset connections properly mysqli_connect('p:localhost', ...)



A3ra%ada3ra4 PHP streams

#ys)lnd uses PHP strea#s portable and robust !ot impleme!tede'pose strea# to users theoreti"lly4 you could i#ple#ent a pro'y



A3ra%ada3ra4 one %ode 3ase

PHP 6 and PHP 7 use the sa#e code base all e'tensions use one code base of #ys)lnd one fi'3 fi'es it all %one bu&3 breaks it all( @nicode is co#piled out for PHP 6

nevertheless the code has only few Jifdef and is readable >f php-net wants #ys)lnd they can &o for one or two branches

PHP 6 PHP 7

#ys)lnd



A3ra%ada3ra4 memory sa!ings


14/421+Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database

A3ra%ada3ra4 read5only !aria3les 02

What e't*#ys)li %or e't*#ys)l3 ---( used with lib#ys)l does4 #ys)li sends a )uery %( result set &et fetched into lib#ys)l buffers %E( #ys)li allocates vals3 then new buffers

%L( #ys)li copies data fro# lib#ys)l to its own buffers calls mysql_ree_result()and deallocates lib#ys)l buffers

What e't*#ys)li used with #ys)lnd does4 #ys)li sends a )uery %( result set is fetched row by row3 every row is different buffer %E( #ys)lnd creates a result set of vals3 pointin& to the buffers

calls mysqlnd_ree_result()which could be li&htwei&ht


15/421,Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database

What e't*#ys)li %or e't*#ys)l3 ---( with lib#ys)l does4 one e'tra allocation for #ys)li buffers one e'tra data copy one e'tra val allocation3

which can be saved with the val cache

Tricks and &oodies of #ys)lnd4 #ys)lnd does thread5safe val cachin& 5

9end $n&ine does this too3 but we brin& it to an e'tre#e0

#ys)lnd records cache statistics %e-&- phpino()output( #ys)lnd collects )uery statistics %#ore later( statistics are per process3 even in a threaded environ#ent

A3ra%ada3ra4 read5only !aria3les 06


16/421-Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database

7ead5only !aria3le im/lementation

zval.type = stringzval.value = *ptr_row1_col1zval.refcount = 1

,ow 5 k L Noe 6 loves F #ys)lnd ---

Misc

zval.type = stringzval.value = *ptr_row1_col1zval.refcount = 2 (or 3)

,ow 5 k L Noe 6 loves F #ys)lnd ---

Misc

,esult set consists of4

ED val bufferresult butter#iscellaneous

val owned by result set4

refcount C

etchin& data4

fast:refcountGG

unlike lib#ys)l always correct hash sie and no rehashin&

unset(ro)4 refcount55

zval.type = stringzval.value = 'Joe'zval.refcount = 1

#ys)liBfreeBresult%(4

Separate %valBcopyBctor%(( all vals for which refcount 3 for e'a#ple4 /oe ro['irstname']



A3ra%ada8sure(84 read5only !aria3les

Me#ory allocators are synchronied synchronisation is slow happens even in sin&le5threaded applications

#ys)lnd allocates bi&&er chunks are we wastin& #e#ory #ini#ies heap fraentation3 less work for the allocator

#ys)lnd tries to reuse the vals saves "P@ cycles less #e#ory fraentation ?ut4 will you notice it


18/421.Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database

9:/ertada3ra4 ;!al %a%he

: On #odule start5up the val cache is initialied- Specific nu#ber ofvals3 ini settin&3 are pre5allocated on a conti&uous block of #e#ory-

: When #ys)lnd needs a val3 it asks the cache3 if the cache has afree entry3 it returns a pointer to one- Otherwise3 allocates a val-

: When a result set is freed3 the vals are destructed throu&h thecache- >f a val is not fro# the cache nor#al 9$ is called- Of coursewe need to separate values3 if refcount -

: >f the val is fro# the cache3 efree%( is not called3 which #eans nocritical sections in the allocator- The val can be used a&ain-

: >f a cache val had refcount 3 it won2t be returned for re5usa&e tillthe end of the script- !i#itation3 which can be lifted in furtherversions by usin& &arba&e collection on ne't free result-


19/421/Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database

9:/ertada3ra4 ;!al %a%he 02

The &lobal %per process( cache consists of4 a pre5allocated val block a free list stack a reference counter

a #ute' other variables for operation and statistics created at M>Q>T

The local %per thread( cache consists of4 a &arba&e collection stack reference counter pointer to the &lobal cache created at ,>Q>T



9:/ertada3ra4 ;!al %a%he 06

: ?ecause co#ple'ity of se)uential scan of the block for findin&free val is O%n(3 the access ti#e would &row with the cache sie-Thus3 an opti#isation called free5list5stack was introduced-

: ree5list5stack holds pointers to free vals3 thus +$T operationhas O%( co#ple'ity-

: P@T is also is also lei&ht5wei&ht because the val has to beadded to the free5list-

: The stack is or&anised backwards- Qew ele#ents &et lower#e#ory addresses- Thus if there is a batch of +$T operationsthe "P@ will cache the line and the first +$T will hit the DATA

!*!E cache-: +arba&e collection at end of re)uest is very cheap as those



9:/ertada3ra4 ;!al %a%he /ro3lems

: ,ecall that on #ys)liBfreeBresult%( the result set has to bescanned and vals with refcount have to be separated-However3 as there is %are( user variable%s( pointin& to a valcache entry3 we cannot perfor# a P@T operation- Thus3 we arestuck with this till the end of the script-

: This #eans that actually with #ys)lnd it2s better if the userdoesn2t call very )uickly mysqli_ree_result()-

: This li#itation can be lifted by tryin& checkin& the &arba&ecollection list of the local cache for vals with refcount C



et%h=all0


25/42

2,Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database


26/42

2-Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database


27/42


9:/erimental4 ..-53ased

$'peri#ental i#ple#entation in the SIQ not part of the released version3 e'peri#ental3 will chan&e0 Ti#e5to5live %TT!( invalidation4 stale data can be delivered On the AP>5level4 si#ple to use3 no e'tra tools needed "usto#ieable stora&e %#e#cached3 file3 ---( planned mysqli_query(strin0 query, boolean cache)

php.ini: mysqli.qc_hash_buer_si1e

php.ini: mysqli.qc_ttl


28/42

2.Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database

.ime5to5li!e 0..- /ro/erties

4KF4KK %&& /0(*) users 445 until 11678677

4KF4K %&& /0(*) users 45 until 11678677

4KF4KE %&& /0(*) users 45 until 116786774KF4KL 9&&& users 7

4KF4K8 %&& /0(*) users 45 until 11678677

4K4KK %&& /0(*) users 75 until 11617677 7

4K4K %&& /0(*) users 75 until 11617677

: TT! is si#ple to i#ple#ent and to use: ,eturnin& stale data is possible

: MyS! %Server( uery "ache never serves stale data

: MyS! uery "ache invalidates on a per table basis

: We drea# of . !o promisewe ever #ake it . per table and perrow invalidation


29/42

2/Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database

A3ra%ada3ra4 Per>orman%e

Ti#e

MyS!Apache "o#pilin& $'ecutin& #ys)lnd

Be!"hmr your prti"ulr ppli"tio! yoursel(5

The AP> i#pact in a web setup can be very s#all it #akes no sense to opti#ie so#ethin& down fro# E to

$'cellent #icro bench#ark results can #elt down !et overall ti#e be Ks and the ti#e spend in lib#ys)l K-s !et #ys)lnd be twice as fast %EKK( as lib#ys)l4 K-K6s ;ou &et down fro# Ks to -6s3 that2s only 6 faster0


30/42


A3ra%ada3ra4 @ell @V@ Store

O!TP application with s#all result sets: Online shop si#ulation3 uses transactions3 re#inds of T"P5"

: Pro/ect pa&e4 http4**linu'-dell-co#*pro/ects-sht#lJdvdstore

: @sed for a contest in which LAM$had by far outperfor#ed others

: "2t contest4 http4**firebird-sourcefor&e-net*connect*ct5db"ontest-ht#l

: "ontest sub#ission4 http4**www-heise-de*ct*dbcontest*teilneh#er-sht#l

: Qot perfect for #ys)lnd3 because of s#all result sets

We found for the contest sub#ission4

: e't*#ys)li with #ys)lnd4 fastest: e't*#ys)li with lib#ys)l4 7 slower

: e't*#ys)l with lib#ys)l4 L slower

mys&l!d is #out s (st s li#mys&l6 sometimes (ster
http://linux.dell.com/projects.shtml%22%20%5Cl%20%22vdstorehttp://firebird.sourceforge.net/connect/ct-dbContest.htmlhttp://www.heise.de/ct/dbcontest/teilnehmer.shtmlhttp://www.heise.de/ct/dbcontest/teilnehmer.shtmlhttp://firebird.sourceforge.net/connect/ct-dbContest.htmlhttp://linux.dell.com/projects.shtml%22%20%5Cl%20%22vdstore


31/42


Sometimes >aster(

DER E66 6DE DKE8 EK8F 8K7 FDE D7LF8 LER7F 76KKK

K

EK

8K

7K

FK

DKK

DEK

D8K

D7K

DFK

#ys)liBselectBvarcharBbuffered-php

!ib#ys)l

#ys)lnd

Percent

!ower bars are faster3 lib#ys)l is blue3 #ys)lnd is purpleetchin& KK rows with one varchar colu#n of a variable sie- Data shown forKK rows3 varchar%n(3 KK' overall-


32/42


'e like this mi%ro 3en%hmark %hart$$$

!ower bars are faster3 lib#ys)l is blue3 #ys)lnd is purple"onnect3 create KK5rows with varchar%#(3 S$!$"T all3 KKK' ti#es #ys)liBdataBseek%#tBrand%((3 close-

DKK'varchar%DEF(

DKK'varchar%E67(

DKK'varchar%6DE(

DKK'varchar%DKE8(

DKK'varchar%EK8F(

DKK'varchar%8DSE(

DKK'varchar%FLF8(

DKK'varchar%D7R7

DKK'varchar%LL6L

DKK'varchar%766K

K

DKK

EKK

LKK8KK

6KK

7KK

RKK

FKK

SKK

DKKK

DDKKDEKK

DLKK

#ys)liBdataBseekBse)uential-php %DKKK rows3 overall(

!ib#ys)l

#ys)lnd

Percent


33/42


.hese are also a%%e/ta3le$$$

DKK row s4 DKKKK' overa ll DKKK rows 4 DKKKK' overall

K

E6

6K

R6

DKK

DE6

D6K

DR6

EKK

EE6

E6K

ER6

LKK

#ys)liBaffectedBrows-php

!ib#ys)l

#ys)lnd

Percent

D KK KK Krows4 o ve ra ll DK KKK Krows4 i ns er tB id %(

K

EK

8K

7K

FK

DKK

DEK

D8K

D7K

DFK

#ys)liBinsertBid-php

!ib#ys)l

#ys)lnd

Percent

E kS$5

6 kS$5

DK kS$5!$"T

LK kS$5!$"T

6K kS$5!$"T

DKK kS$5!$"T

6KK kS$5!$"T

6KKK kS$5!$"T

DKKKKk S$5!$"T

K

EK

8K

7K

FK

DKK

DEK

D8K

D7K

DFK

EKK

#ys)liBfetchBlon&Bbuffered-php %DK rows3 overall(

!ib#ys)l

#ys)lndPercent

6KKKKK'overall

DKKKKK'overall

DKKKK'overall

DKKK' overall DKK' overall

K

6K

DKK

D6K

EKK

E6K

LKK

L6K

8KK

86K

6KK

#ys)liBrealB)uery-php

!ib#ys)l#ys)lnd

Percent


34/42

3+Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database

can be very s#all

--- lib#ys)l and #ys)lnd run thousands of op- per second

--- fro# K-KKKs to K-KKKKLs often #akes no difference

--- we know we #ade it faster3 the charts are real3 butcheck twice if you really profit fro# #ys)lnd

perfor#ance i#prove#ents

--- if you really want #ore charts4 we are preparin& a LKpa&es perfor#ance docu#ent

Typically we found456 to G6-

?ut really no pro#ise0


35/42

3,Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database

7oadma/

: "o#plete the work . be there in ti#e for PHP 7 SS! support "o#pressed protocol support write docu#entation3 educate users

: Support PDO*MyS! Qobody has volunteered for the task--- --- looks like we have to do it

: Move develop#ent and source to php-net Of course4 only if the PHP fellows want #ys)lnd on php-net Of course4 not before its stable enou&h not to break PHP---


36/42

3-Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database

.he >uture is yours

;our wishes are4

Hands up3 please0 There is plenty of roo# on this slide0 We can add #ore slides with your ideas and wishes0


37/42


Possi3le >utures$$$

: Prepared state#ents "ache handles4 prepare is e'pensive3 e'ecute is fast Si#ilar to persistent connections or val5cache

: "lever client5side )uery cache TT!5based invalidation as a default ,e#ote and confi&urable stora&e of cache entries Drea#4 MyS! uery "ache style invalidation Drea#4 row5level invalidation

$'pose PHP strea#s to users Transparent Pro'y Translatin& Pro'y


38/42

3.Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database

?dea to try4 3a%kground >et%h

PHP scripts run slow co#pared to " pro&ra#s Server4 #e#ory and locks are released faster3 lowers overall load

"lient4 reduces latency3 SMP support3 works with prefork and threads

Send )uery $'ecute )uery

"ontinue script

Start fetchin&

"reate*invoke back&round

Deliver resultsPrefetch all %asynchronously(

Process row

ree result set

,elease locks

etch row E

Deliver data

Process row E

Deliver dataQetwork

Dear Presenter3do not #ention that this is new to the PHPworld- We have no e'perience with it-

Deal !istener3do not ask for the first perfor#ance fi&urewe saw when tryin& the idea out in a hack-

>t is an idea3 no #ore0>t #i&ht never be i#ple#ented0


39/42

3/Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database


40/42

+0Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database

Some randomly sele%ted Bontri3utors

: !ukas ahwe S#ith AP> ideas and #ore--- http4**pooteeweet-or&*

Peter 9aitsev Statistics3 back&round fetch3 bench#arkin& hints and #ore--- http4**www-#ys)lperfor#anceblo&-co#*about*
http://pooteeweet.org/http://www.mysqlperformanceblog.com/about/http://www.mysqlperformanceblog.com/about/http://pooteeweet.org/


41/42

+1Copyright 2007 MySQL AB The Worlds Most Popular Open Source Database

uestions (

Ok3 you do not dare to ask-However3 at any ti#e3 #eet

#e on >,"3 ask #e by #ail3 ---


42/42

Ceed3a%k, /lease

Avera&e . +ood . $'cellent . Hands up0

My overall ratin& for the talk is--- This talk was too difficult to understand---

This talk did not &ive enou&h technical infor#ation---

>f > were you3 > would chan&e this--- . raise your voice0

Thank you and &ood bye0

Documents

Andrey Mysqlnd