of 43 /43
scaling LAMP doesn’t have to suck! @stevencorona (400 million api calls/day) (3 billion photos)

Steve Corona: Scaling LAMP doesn't have to suck

  • Author
    webexpo

  • View
    3.364

  • Download
    0

Embed Size (px)

DESCRIPTION

"I went from sleeping in front of my laptop, restarting apache every hour, to automatically scaling servers from my phone." More at http://webexpo.net/prague2013/talk/scaling-lamp-doesnt-have-to-suck/

Text of Steve Corona: Scaling LAMP doesn't have to suck

  • 1.scaling LAMP doesnt have to suck! @stevencorona (400 million api calls/day) (3 billion photos)

2. lets learn how to scale php ...in 40 minutes 3. scaling is... architecting your app for millions of users 4. scaling is... an art, not a science 5. scaling is... NOT CODE about infrastructure your code is fast enough 6. LAMP LINUX APACHE MYSQL PHP 7. problems single points of failure apache+mod_php memory hog slow linux default settings cant do work in the background 8. a better LAMP LINUX +TUNING NGINX PERCONA PHP-FPM NSQ REDIS 9. nginx loadbalancer nginx php-fpm MySQL Master nginx php-fpm nginx php-fpm nsq MySQL Read Only redis 10. tuning linux linux is mostly sane use latest kernel distro doesnt matter (i like ubuntu) 11. top 3 settings open_les network/sysctl tuning i/o scheduler most settings you nd online are outdated 12. open les default is 1024 on linux, each socket = open le /etc/security/limits.conf soft nofile 999999 hard nofile 999999 13. sysctl tuning thousands of settings newer kernels autotune network /etc/sysctl.conf net.core.somaxconn=999999 net.ipv4.ip_local_port_range=2000 62000 net.ipv4.tcp_tw_recycle=1 vm.swapiness=0 14. i/o scheduler linux has swappable scheduler default is cfq $ echo deadline >/sys/block/sda/queue/scheduler +50% IOPS on MySQL 15. nginx open-source HTTP server swap out apache easy + sane cong nginx.org 16. nginx most popular webserver of top 1000 sites 15,000 requests/second with 20% CPU and100MB RAM 17. nginx 18. nginx load balancer http cache fastcgi proxy web server 19. php php-fpm is king standard with php5.3 20. php-fpm use static worker pool 4x number of CPU cores pm=static pm.max_children=128 21. php php 5.5 is much faster than 5.4 and 5.3 USE IT! 22. php php 5.5 includes Zend Optimizer+ (no more APC) 10-20% faster than APC in real world 23. php horizontal scaling watch out for sessions! 24. php + mysql mysql persistent connections are GOOD to use. despite what the internet says 25. php + mysql 20-25% faster for rst query after that it doesnt matter 26. php + mysql watch out for thundering herd mysqls max_connections 2(pm.max_children * # servers) 27. php + mysql turn on mysql protocol compression $m=mysqli_init(); $m->real_connect($username...., MYSQL_CLIENT_COMPRESS); 28. percona open source mysql fork patches from twitter and google percona.com 29. percona drop-in replacement faster at scale, more reliable 30. percona new transactions per minute (more is better) 31. percona only use innodb disable query cache enable thread pool thread_handling=pool-of-threads 32. percona watch out for NUMA innodb_buffer_pool_populate=1 flush_caches=1 numa_interleave=1 33. percona stop wasting time use SSDs went from15 mysql servers to 2 using SSD 34. redis in the old days, we used memcache redis is better redis.io 35. redis data persisted to disk hot cache 500,000 GET/second 36. redis cong is great out of box avoid stalls on EC2 use HVM instances 37. redis lots of php libraries use pecl-redis c extension pecl install redis 38. nsq message queue server in golang by bit.ly we use it to process 370 million events/day https://github.com/bitly/nsq 39. nsq nsqphp is best php library pub/sub model do work async api calls defer long-running work https://github.com/davegardnerisme/nsqphp 40. nsq 41. nsq DONT USE MYSQL AS A WORK QUEUE! 42. dont scale code your code is fast enough scale infrastructure 43. thanks! follow me on twitter @stevencorona