9gag Clone

D N 9GAG CLONEI. Yu cu Pht trin 1 website vi cc chc nng nh 9gag.com Nhanh, n gin, p p ng s lng ln ngi truy cp ng thi C kh nng m rng, phn tn, phn ti C kh nng m rng sang ng dng trn smartphone a ngn ng SEO tt Bo mt cao Code trong sng, comment y

II. nh hng Vit phn core ring bng NodeJS, phn giao din ring, giao tip qua API m bo c th ti s dng khi chuyn qua smartphone Phn giao din s dng PHP, m hnh MVC, framework t to Database MySQL S dng Redis nh 1 database ph S dng nhiu webserver ti u cho tng nhim vCc webserver s dng cc tn min khc nhau, nh th s d dng cho vic phn ti, tch server

-

S dng Sphinx cho tm kimNodeJS ang l cng ngh rt hot hin nay. T lc ra i (2009) n lc hot node.js ch mt khong vi thng. iu ny c l ch c th gii thch bng l do l vit c client side v ln server side ca cng mt trang web ch bng mt ngn ng l c m ti cao ca gii lm web.

http://cntt.tv/nodes/show/575 Nginx (pht m "engine x") l mt chng trnh server HTTP, mt reverse proxy cng nh IMAP/POP3 proxy server min ph, m ngun m, c hiu nng cao. Nginx c tc gi Igor Sysoev vit cho rambler.ru , trang web c truy cp nhiu th 2 trn nc Nga, chy n nh trong vng hn hai nm ri.

http://cachcaidat.com/2011/05/tin-t%E1%BB%A9c-nginx-1-0-0-ram%E1%BA%AFt-sau-9-nam-phat-tri%E1%BB%83n/Nghim Tin Vin boygiandi!

Page 1

Sphinx l search engine tc cao c ting hin nay. Mt s trang web ln ca Vit Nam nh RngBay.com s dng thnh cng Sphinx. Trc y Sphinx ch h tr SQL DB nh MySQL v PostgreSQL, t phin bn 0.9.9-rc2 tr i Sphinx h tr thm XML, gip Sphinx m rng th trng ra rt nhiu.

http://cntt.tv/nodes/show/536 Redis c th coi l 1 nosql database (hay key-value store) chy trn RAM. N thng c s dng cha cc d liu c tn sut truy cp cao hoc d liu thng xuyn bin i. Cc website c lng ngi s dng ln thng ng dng Redis m bo tc c d liu.

http://blogs.go.vn/viewpost/43314915/144896/redis-giai-quyet-mot-so-van-dethuong-gap-khi-lap-trinh-web-part-1-.htm Varnish l trnh tng tc HTTP hot ng nh mt load balancer (b phn ti) v h thng cache ni dung gip tc ti trang l nhanh nht. Facebook s dng Varnish x l nh thng thng v cc nh trong profile ngi dng. N c th x l hng t requests (yu cu) trong mt ngy. Tng t cc phn mm khc c Facebook s dng, Varnish cng l mt phn mm ngun m.

http://forum.szone.vn/threads/1357-Tim-hieu-nhung-cong-nghe-su-dung-trenFacebook?s=8f9aad5ce8b9842db575f166044e7838 HAProxy is a free, very fast and reliable solution offering high availability, load balancing, and proxying for TCP and HTTP-based applications. It is particularly suited for web sites crawling under very high loads while needing persistence or Layer7 processing. Supporting tens of thousands of connections is clearly realistic with todays hardware. Its mode of operation makes its integration into existing architectures very easy and riskless, while still offering the possibility not to expose fragile web servers to the Net.

http://haproxy.1wt.eu/ G-Wan l 1 webserver vit bng C c qung co l nhanh vt tri so vi tt c cc loi khc. Ci ny cn t ngi s dng nn mnh th lun.

http://www.wikivs.com/wiki/G-WAN_vs_Nginx http://loadstorm.com/2011/web-server-performance-benchmarks

-

-

-

-

Ngi dng website ch yu l xem nh, do m bo hiu nng h thng, ta s tch cc request file tnh (nh, css, js..) i 1 ng v request PHP i 1 ng. Dng HAProxy ng trc (cng 80) lm reverse proxy v cn bng ti sau ny, chuyn cc request PHP cho Apache, vi cc request tnh th dng Nginx hoc Varnish hoc GWan Apache kt hp vi PHP tt hn cn Nginx th x l file tnh tt hn. Varnish v Nginx v GWan ci no tt hn th s ci c 3 ri th nghim. NodeJS l 1 Server-side scripting ( nh PHP ) c giao din ngn ng (syntax) l JavaScript do cc k quen thuc i vi ngiPage 2

Nghim Tin Vin boygiandi!

-

-

-

-

pht trin web, rt d pht trin v c bit l tc nhanh hn PHP. Do mnh s dng NodeJS vit phn core. Phn core v phn giao din PHP s giao tip vi nhau qua HTTP API. 1 ng vin khc c th m nhn v tr ny l Python, nhng mnh vit Python khng quen bng NodeJS. Phn core m nhim vic kt ni v x l truy vn ti database mysql, phn giao din mun tng tc vi database phi gi qua API. Search c th dng Solr hoc Sphinx, Solr mnh dng qua nn ln ny th dng Sphinx. Nu khng c 2 ci ny c th dng Full text search sn c ca MySQL nhng cht lng khng bng. Imagick dng cho x l nh bao gm resize, crop, gn watermark File nh lu trn cng. Th mc cha nh c chia lm 3 cp. VD : upload/0/0/file.jpg . Th mc upload/0/0/ s cha 1024 file, s lng s t ng to th mc tip theo (upload/0/1/ ) Th mc upload/0/ cha ti a 1024 th mc con t 0-1023. Th mc upload/ s cha 1024 th mc con t 0-1023. Nh vy s cha c ti a 1024 * 1024 * 1024 = 230 file. Gi s mi ngy ngi dng upload 1024 nh th s mt 2.873 nm dng lp y cc th mc ny Hm lu nh vo cng cn c thit k d phng cho trng hp sau ny tch server cha nh thnh 1 server ring. Redis s c dng lu danh sch id cc bi vit. S d nh vy l bi hm lit k bi vit ca chng ta cn ly trong 1 khong th cu lnh truy vn MySQL s ~ SELECT * FROM `post` LIMIT start,count , tc thc hin truy vn ny s rt chm khi start ln v table ln. Redis lm vic ny tt hn nhiu. Chng ta s dng Redis ly id bi vit trong khong (start, start+count) ri dng MySQL c ni dung bi vit. Cc li bo mt thng gp

Hng tn cng

Cch phng trnh

SQL Injection XSS Upload ScriptNghim Tin Vin boygiandi!

- Kim tra u vo - S dng MySQL Procedure - Kim tra u vo - Kim tra file upload ln ng l file nh th mi thng quaPage 3

http://khanh.com.vn/post/2011/10/03/Bao-mat-web-server-Apache-voi-modSecurity-Phan-3.aspx

III. D kin cc cng ngh s dng

Apache PHP MySQL Varnish Imagick Nginx G-Wan HAProxy Sphinx NodeJS Redis M hnh mng

Webserver

Cache file nh ngi dng X l nh Webserver for static file Webserver for static file Proxy cn bng ti Search Vit webservice Dng nh 1 database ph


Page 4

IV. Ci t mi trng 1. Ci t cc phn mm c bnyum install make pcre-devel libtool gcc gcc-c++ screen glibc glibc-devel libgcc sqlite sqlite-devel

2. Ci t Apacheyum install httpd httpd-devel /etc/init.d/httpd start Nghim Tin Vin boygiandi!

Page 5

Config apache chy cng 81, restart 3. Ci t PHPyum install php php-mysql php-common php-gd php-mbstring phpmcrypt php-devel php-xml /etc/init.d/httpd restart

4. Ci t MySQLyum install mysql mysql-server mysql-devel /etc/init.d/mysqld start mysql

5. Ci t Imagickyum install ImageMagick* yum install php-pear pecl install imagick /etc/init.d/httpd restart

6. Ci t Nginxwget http://nginx.org/download/nginx-1.0.12.tar.gz tar -vxzf ./nginx-1.0.12.tar.gz mv ./nginx-1.0.12 ./nginx cd ./nginx ./configure make make install

7. Ci t Varnishwget http://repo.varnish-cache.org/source/varnish-3.0.2.tar.gz tar -vxzf ./varnish-3.0.2.tar.gz mv ./varnish-3.0.2 ./varnish cd ./varnish make make install

Config varnish ( varnish.vcl )Nghim Tin Vin boygiandi!

Page 6

backend apache { .host = "localhost"; .port = "81"; .connect_timeout = 600s; .first_byte_timeout = 600s; .between_bytes_timeout = 600s; .max_connections = 800; } acl purge { "localhost"; } sub vcl_hit { if (req.request == "PURGE") { set obj.ttl = 0s; purge; error 200 "Purged yyyyy."; } } sub vcl_miss { if (req.request == "PURGE") { purge; error 200 "Purged xxxxxxxxxxxx."; } } sub vcl_recv { if (req.http.host ~ "cdn1.xxx.net") { set req.http.host = "cdn1.xxx.net"; set req.backend = apache; if (req.request == "PURGE") { if (!client.ip ~ purge) { error 405 "Not allowed."; } return (lookup); } # Cache things with these extensions js|css| if (req.url ~ "\.(jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf)$") { unset req.http.cookie; return (lookup); } else if (req.url ~ "\.(js|css)$") { return (pass); } else { error 405 "Not allowed."; } return (lookup); } else { return (pipe); } } sub vcl_fetch { if ( beresp.status != 200 ) { set beresp.ttl = 0s; Nghim Tin Vin boygiandi!

Page 7

} if (req.url ~ "\.(jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf)$") { set beresp.http.magicmarker = "1"; } } sub vcl_deliver { if (resp.http.magicmarker) { unset resp.http.magicmarker; unset resp.http.Cache-Control; set resp.http.Cache-Control = "max-age = 2678406;public"; set resp.http.Expires = "Thu, 16 Feb 2013 12:24:25 GMT"; set resp.http.Age = "0"; } }

Start varnish, chy cng 82, 2G RAMvarnishd -a :82 -T localhost:6082 -f /var/www/config/varnish.vcl -s file,/var/cache/varnish.cache,2G

8. Ci t Sphinxwget http://sphinxsearch.com/files/sphinx-2.0.3-release.tar.gz tar -vxzf ./sphinx-2.0.3-release.tar.gz mv ./sphinx-2.0.3-release ./sphinx cd ./sphinx ./configure --prefix=/usr/local/sphinx make make install

9. Ci t NodeJSwget http://nodejs.org/dist/v0.6.10/node-v0.6.10.tar.gz tar -vxzf ./node-v0.6.10.tar.gz mv ./node-v0.6.10 ./nodejs cd ./nodejs ./configure make make install

Ci modulecd ./nodejs npm install express Nghim Tin Vin boygiandi!

Page 8

npm npm npm npm npm npm

install install install install install install

db-mysql generic-pool mysql-libmysqlclient hiredis redis winston express-request-logger

Mt s module phi ci bng tay https://github.com/kurokikaze/limestone

10.

Ci t Redis

wget http://redis.googlecode.com/files/redis-2.4.7.tar.gz tar xzf redis-2.4.7.tar.gz mv ./redis-2.4.7 ./redis cd ./redis make screen S redis src/redis-server Ctrl+A Ctrl+D src/redis-cli config set maxmemory 524288000 quit

11.

Ci t gwan

wget http://gwan.com/archives/gwan_linux64-bit.tar.bz2 tar -xjf gwan_linux64-bit.tar.bz2 cd gwan sudo ./gwan

12.

Ci t HA Proxy

http://linuxadminzone.com/how-to-install-setup-and-config-haproxy-loadbalancerfor-content-switching/

V.

Pht trin phn mm - Tnh hot trang ch website c 1 tab l Hot topic, vy cn 1 hm nh gi th no l 1 hot topic. Nh vy cn xy dng 1 hm tnh im cho tng bi, bi no im cao th trn (hot hn). im s phi ph thuc vo lng like, comment, ph thuc thiPage 9


-

gian n c th tri i c (trnh trng hp 1 bi l hot th n c hot mi) cng thc : (like+comment)/time^alpha Cn vit 1 chng trnh, 15 pht chy 1 ln, ly 1000 bi mi nht t redis ra tnh hot. Cp nht li vo redis. Ti u : Cng c ti u hiu qu nht cho PHP l XDebug. Vi xdebug ta c th bit c tng cu lnh 1 chim bao nhiu thi gian, t ti u nhiu nht c th. Tuy nhin Xdebug cng lm PHP x l chm i. Do khi ti u xong ri th cng nn disable xdebug i.Ci t: http://xdebug.org/docs/install

Cu hnh:xdebug.auto_trace=On xdebug.profiler_enable=On xdebug.profiler_append=On xdebug.profiler_enable_trigger=On xdebug.trace_output_name= trace.%R

File log s c lu vo /tmp , bn dng phn mm WinCacheGrind c file ny


Page 10

-

Benchmark : S dng cng c Apache Benchmark.http://httpd.apache.org/docs/2.0/programs/ab.html$ab -n 1000 -c 100 http://******.net/ This is ApacheBench, Version 2.3 Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking goodmeme.net (be patient) Completed 100 requests Completed 200 requests Completed 300 requests Completed 400 requests Completed 500 requests Completed 600 requests Completed 700 requests Completed 800 requests Completed 900 requests Completed 1000 requests Finished 1000 requests Server Software: Server Hostname: Server Port: Document Path: Document Length: Concurrency Level: Apache/2.2.15 ******.net 80 / 17058 bytes 100


Page 11

Time taken for tests: 2.797 seconds Complete requests: 1000 Failed requests: 7 (Connect: 0, Receive: 0, Length: 7, Exceptions: 0) Write errors: 0 Total transferred: 17168778 bytes HTML transferred: 16996778 bytes Requests per second: 357.49 [#/sec] (mean) Time per request: 279.728 [ms] (mean) Time per request: 2.797 [ms] (mean, across all concurrent requests) Transfer rate: 5993.83 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median Connect: 0 1 1.5 0 Processing: 29 270 184.9 201 Waiting: 28 269 184.9 200 Total: 36 270 185.6 201 max 7 1021 1021 1028

Percentage of the requests served within a certain time (ms) 50% 201 66% 211 75% 216 80% 222 90% 551 95% 805 98% 905 99% 941 100% 1028 (longest request)

- Thi gian thc hin d n : 3 tun VI. Kim th 1. Tiu ch o Ti bt k trang no u di 200ms (ging tiu ch ca Zing) o CPU thng xuyn di 10%, max khng qu 30% o I/O Wait di 5% 2. Kt qu Cha o c Performance do ngi dng cha ng VII. Bo tr


Page 12

Documents

9gag Clone