140, Bd Haussmann 75008 Paris · Standard +331.58.56.60.80 · Fax : +331.58.56.60.81 · Web : SAS au capital de 203 905 € · R.C.S. Evry

140, Bd Haussmann 75008 Paris · Standard +331.58.56.60.80 · Fax : +331.58.56.60.81 · Web : www.nbs-system.com

SAS au capital de 203 905 € · R.C.S. Evry B 423 410 901 · N° TVA : FR 61 423 410 901

Make Magento run like an antelope

01.58.56.60.80 / [email protected]

►Bio►About Magento►Speed matters, a LOT►From 10s to 3s►Breaking the 1.5s wall►What servers, what volumes



• I’m Philippe, 35, founder of NBS System 11 years ago• Community Advisory Board Member• Creator of Wikigento & Bargento (FR community member)• Kind of a tech guy mixed with a commercial (and a bit of beer) • Used to be a pentester, a CTO, a CCO, then the bizdev guy to put it short

Short Bio - me

• Half security (pentest - audit)/ Half managed hosting• 280 hosted Magento sites, 150 servers dedicated to it, french leader• 17 persons, all located in France

Short Bio – my company



Magento is oftenly considered like

• Powerfull• Solution of the futur• Complicated to code• So much fun !

And

• Slow• Ressources hungry• CPU intensive• Complicated to code

Well, this is true, most of it.



Organisation de la productionNow why Magento can be more

likeFirst consider the wins

• More sales• More efficient• More flexible• Cost less to modify /

admin• Will last (22 M$ should

help)

Then the drawbacks• Ressource intensive• A bit tricky to masterThen the solution

Put a better hosting environment to correct the « fat boy » feeling and only appreciate good points.



Organisation de la production

►Google is now using speed as an important ranking factor (it’s official)

• Graph (given by Google) show clearly the target: 1.5s• Actually, 1,5s means you’re part of the 75% fastest sites but…• More time and you are part of the red team in the red zone, called

« SLOW »• Guess what, this is no good for your ranking & SEO

BesidesRevelance is still the most important criteria among 200, but, http://ow.ly/1xfqt

►Amazon loose 1% revenue every 0,1s latency►Your customers feel it, this is part of their experience on your site !

Our Goal is now clear : load in less than 1.5s

Speed maters, a LOT

http://ow.ly/1xfqt



1. The right hardware2. The right OS3. An efficient architecture4. Code like a God

The keys

Stepping down from 10+ to 3 seconds




• Hardware manufacturer have very different efficiency • Intel and AMD processors doesn’t behave the same• Hyperthreading changes (almost) nothing in server performances• RAM make a HUGE difference on I/O

HARDWARE


►Xeon for front servers, 5670 if possible►AMD mono - hexacores for DB►Use DDR3►Use SAS controllers with cache►Put a LOT of RAM in them (we’ll see later

why), 2 Go per core !

Advices




• Being OS Agnostic, I’m objective• Windows consume too much ressources on PHP interpretation,

weak TCP/IP stack, not optimized for *AMP environment• OpenSolaris gave poor results (30% less than Linux)• NetBSD behave well, very fair competitor• Linux is leading quite seriously

Operating System


Advices►Debian Lenny►CentOS►NetBSD►Redhat(avoid the GUI based distro)




• A Firewall + Loadbalancer• A Reverse Proxy will offload the servers• Content server for static files (CSS/JPG/GIF/JS…)• You can deliver satic content « near to the user » with CDN• Separated web & database servers (+33%)

Create an efficient architectureStepping down from 10+ to 4 seconds

Advices►Netfilter for FW►IPVS + keepalived►NGINX for the RP►Apache, Litespeed, Zeus, Nginx

Conservative ? Apache + SquidR&D guy ? Nginx + PHP FPM + Mango DB

Internet

Nom : Ivoire

Rôles : - Firewall - DNS interne - Reverse Proxy - Load Balancer

Plateforme mixte production et recettes

Plateforme de production

Serveur Web

Base de données

Serveur Web Serveur Médias et moteur de recherches

Base de donnéessecondaire

Synchronisation temps réel des bases

Sauvegardes

DNS

Mails

Backend Magento

Frontend Magento via serveurs Web




Code like a God


Code maters. A HUGE LOT.

Each little glitch is going to be amplified by the visitor numbers : Small site : dirty things won’t be too much of a problem Big site : a miscoded loop can take down the servers

Be helped by community►Fooman Speedster, Magento boosters (like delorum’s one)►Read Forums, blogs►Bargento (to listen to our top notch talks)►Apply the whitebooks and blog post optimisation tips

Be helped by Magento Inc►A bit of consulting by the pro, a lot of time won !




Code like a God


►Go to a strong training course►Understand the block caching►Take care of the security ! (xss/csrf/sql-injection)►Leaving a 404 on a picture create a PHP exec : bad !

►Optimise the CSS selectors & the CSS calls (just needed one)►Serialize CSS and then JS order in the downloads (not css/js/css/js)

►Minify your JS using Jsmin►Avoid redirections, optimize pictures, and DNS queries

►Be carefull while in a loop, think that your piece of code will sometimes be executed thousands of time a minute…




1. Get rid of the major bottleneck : I/O2. Down deep in the mine3. Advanced Tweaks4. Analyse your trafic5. Use cache the right way6. More ?

a.k.a « Where the fun starts »

Breaking the 1.5s wall !




The I/O Bottleneck


Advices►Put sessions & cache on tmpfs ramdisks►Use fast disks like 15Ktm SAS and a good

controller with RAM to boost the writes►Have your disks in RAID 10 if possible►Don’t log useless things (& don’t log-resolve)►Lot of RAM ? Put the whole site on !

►I/Os are a major problem►Write to disk, but with a good Controller, on a excellent disk►Read from disk == loose time (except on VERY good SAS controllers)

►ALL TO RAM ! We’ve got plenty of it !o Faster and more reliable than a SSDo Optimized by CPU & BUSo Inexpansiveo Dawn fast




Split read & write to different DB

<default_read> -> host it on a RAMDISK<default_write> -> Host it on a Physical Disk

Safety + Performances

Use DRBD

DRBD is an opensource Blockdevice sharing over network. Mainline in Linux Kernel since 2.6.33 => safe

Simple, safe, efficient (and made for High avaibility environments). This give good results and can be used but it’s limited to only two servers for now.

MYSQLStepping down from 10+ to 4 seconds



Organisation de la productionAdvanced Tweaks

►Use Sysfence to autokill the bombing crons or php process►Use IRQ_Balancing►Mount your disks with noatime option (faster access to data)►Lower socket Wait Timeout in /proc►Truncate (or archive) your log tables in Magento►Check the best scheduler, iomapper, L3 (architecture dependant)►Use GRsec/Pax (for security purposes)►Tweak Zend server rules►Optimize your peering map (network)►Put a QoS tag on your packets to speed them up►Check foreign includes response times and put them at the end of pages►Use pagespeed / Yslow / Gtmetrix to analyse your site►Integrate Lucene SOLR on large catalog database (> 100 000 products)►Use latest version of Magento and activate all caches in the BO (if possible)





Analyze your trafic (know your ennemy)

Advices

►Use your mailing wisely, spread it on some hours►Same for all trafic generation, « burst » is your ennemy

0

5000

10000

15000

20000

25000

30000

35000

40000

45000

50000

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

Nom

bre

de c

onne

ctés

Ventilation horaire

Répartition type d'un site de vente en ligne sur la base de 400 000 VU/jour Typologie: Textile/chaussure femme

CloudArchitecture





Use Cache the right way

Tips & Tricks

►PHTML blocks should be cached it in order not to be generated each view►To cache a block, you have to declare it explicitly, no « auto mecanism »►Developers usually don’t use the Block caching mecanism at all (80%)

►Up to ten times faster loading when using cache properly !

►The Enterprise Edition has a full page caching mecanism (for the final pages, not the individual blocs)

►NBS will soon release a plugin that will « automagically » do it for you





o Funkload & APDEX are the keys to « realistics » benchmarks

o When you reach a 0,9 APDEX indice, your server has given enough power and your user still feel confortable.

o 4 levels for user experience (like your girlfriend, except for the timings) :

o Satisfied (for exemple 3s)o Tolerating (3x more, so less than 9s)o Frustrated (more than 9s)o Inacceptable (not served/errors)

Benchmarking



Organisation de la productionBenchmarking




o With 5670 dual quad core servero 8 Go of DDR3 1066 Mhzo Optimised settings & serverso Database splited to another servero Very good site (optimized)o 80 000 productso Magento 1.4.0.1 + FC demostoreo APDEX Limited to 0,9

80 000 uniq visitor / day per front server650 Concurrent Users

0,67s average load time on the home(it’s still a « perfect case », count more on 50 KVU)

What servers, what volumes ?



O

Thank you for your attention !

NBS System140 bd haussmann

75 008 ParisTel : +33.1.58.56.60.86

Mail : [email protected] : www.nbs-system.com

01.58.56.60.80 / [email protected]

Need (fast) hosting ?

8 november 2010, PARIS750 + visitors, 30 booths16 talks

mailto:[email protected]

http://www.nbs-system.com/

Documents

140, Bd Haussmann 75008 Paris · Standard +331.58.56.60.80 · Fax : +331.58.56.60.81 · Web : SAS au capital de 203 905 € · R.C.S. Evry