Mindtalk Tech - Behind the scenes

Mindtalk Technology

Behind the Scenes

Robin Syihab (CTO Mindtalk)MT: @robinTW: @anvie

Apa itu “Scalability”?

“Scalability” bukan

•Performa yang tinggi.

•Mudah untuk menambahkan server.

•High availability.

Skalabilitas adalah

•Kemampuan dalam mengakomodir peningkatan permintaan yang tinggi.

•Bukan hanya masalah server atau teknologi, tetapi juga sumber daya manusia dan proses di dalamnya.

Bagaimana Mindtalk

menghadapi masalah

skalabilitas?

Database

•MongoDB (main database).

•Redis (cache).

•Tokyo Cabinet (url shortener).

Database

MongoDB(master)

MongoDB(replica)

Service

Cache

Write

HAProxyRead

MongoDB(shard)

MongoDB(shard)

MongoDB(shard)

MongoDB(shard)

Web Server

•Nginx

•HAProxy (failover)

Web Service

web service

rest API service

Mobile web service

consumer

HAProxy

Tips Optimasi

•Selalu aktifkan gzip.

•Jangan gunakan domain yang sama untuk web dan serving static files.

•Tuning web level cache.

•nginx workers == jumlah core.

•Jika perlu gunakan varnish.

Elastic Search

•Mudah menambahkan shard, tidak perlu servis tambahan seperti servis kordinator, contoh: Zookeeper.

•Skema dokumen yang flexibel.

•Faceting.

•Pencarian ke banyak index.

•Automatic shard rebalancing.

Development Process

Source Control Management : GIT

Continuous Integration: Buildbot

Code Inspection: Gitlab

Monitoring System: PandoraFMS

http://www.lafermeduweb.net

Deployment System

Makefile increment versionpre-build test

build distribution

backup

staging

updating

Logging

Levels:

DEBUG

INFO

WARNING

ERROR

file A (rotated)

file B (rotated)

file C (rotated)

email to maintainersincludes technical information

Bahasa pemrograman

•Python - Backend services.

•Scala - Stream processing.

•Node.js - Realtime notifier.

•PHP - Front end.

FRONT ENDS

Closure Library

GoAnscdn & AWS

Sock API

web API Cache

Cache

Search Engine

Hadoop

Database

Database

Notifier

Replica

•Authentication•API

Message Queueing

•Mobile Push Notification

•Web Notificationlong polling

Data mining storage

MID ENDS BACK ENDS

•3 instances round-robin by hproxy

•Includes Lucene Engine

•Purpose to:• Generate analytics.• Calculate search relevancy.• Calculate suggestion like PYMK or GYML.

Ndayak 3 EngineNdayak 3 Engine

Geek Wanted

•Java / Scala Programmer.

•NoSQL Database Engineer.

•System Administrator.

kirimkan CV ke:job@mindtalk.com

Terima kasih