Tối ưu hóa hệ thống 30 triệu người dùng

IT Expert Club

CONTENT

1. Introduction2. Solution3. Caching4. Asynchronous5. Scale 6. Distributed7. Profiling and Turning

1. Introduction

CTO of Ecomobi

Contact info:• EMAIL: TUYENDV@ECOMOBI.COM• MOBILE: (+84)904.554.112• SKYPE: ITBACHKHOA

1. Introduction

Ecomobi is a CPA Network focus on SEA Market.

Main Flow

Daily Information

Affiliate Networks• 1.5M+ DAU• 2-3M CLICK • 1TB TRANSFER• USER: VN, ID, IN, BR...

Ads Networks• 30M+ USERS• 1M+ DAU• 40M+ ADS REQUEST• 70K+ RPM• USERS: VN, ID, IN,

BR...

Our Issues

• High Availability• Minimize Response time• Concurrency • High Bandwidth• International user

2. Our Solutions

• Measurement tools• Select Services Provider• Programming language• Database• Others

2. Measurement

• Analytics• Quick or Slow• Error or Accuracy• Stable or Unstable• Better or Worse

2. Measurement Tools - New Relic

2. Measurement Tools - New Relic

Services Provider?

• Dedicated Server in Viet nam• Private cloud• Docker

• VPS Provider in Vietnamese• Amazon Web Service• Digital Ocean / Linode• VPS in Local Provider

Programming language?

• PHP• JAVA• Erlang / Elixir• Node JS / Golang

Database

• MySQL / MariaDB• PostgreSQL• MongoDB• Cassandra / couchbase / Riak• Redis / Memcache • Elasticsearch

Message Queue

• RabbitMQ• Redis• Activemq / Zeromq• Kafka• Gearman (Job Server)

Other

• Caching• Asynchronous• Scale out• Distributed

Our Stack

3.Caching

• In-memory / variable caching• Cache size => minimize (why) • Increasing Cache Hit / Cache Miss• Cache Read through issue

3.Caching: memory

function getInfoWithCache($id) {

static $cache_contents = array();

if (isset($cache_contents[$id])) {

return $cache_contents[$id];

}

$data = getInfoWithoutCache($id)

$cache_contents[$id] = $data;

return $data;

}

Caching - Minimize cache size

3.Caching - increasing cache hit

$candidates = Campaign for

• Active campaigns

• Allow Android 4.0.1

• Allow Vietnam

• Not gambling

• CPI or CPC• IF CPC: PRICE > $0.03• IF CPI: PRICE > $0.5

• Did not install this game / app

…

Candidates = Campaign for

• Active Campaigns

• Allow Vietnam

Candidates = Filter(Candidates)

• Allow Android 4.0.1

• Not gambling

• CPC or CPC

…

Cache read through issue!!!function getData($id){ $data = getDataFromCache($id); if (!empty($data) { return $data; } $data = getDataWithoutCache($id); setDataToCache($id, $data, $exprire_time); return $data;}

4. Asynchronous

• Send a external HTTP request

• Send a email

• Working with file

• Working with DB

• …

4. Asynchronous

• Decreasing Response time• Deal with huge of request

4. Asynchronous - MQ

4. Asynchronous - MQwhile (true) { $msg = getMessageQueue(); if (empty($msg)) { sleep(1); continue; } processMQ($msg); if ($need_stop) { break; }}

5.1 Web Application Scale

5.2 Database layer

6. Distributed

6. Distributed

6. CDN

7. Profiling