Embed Size (px)
DESCRIPTION
Mindtalk Technology - behind the scenes
Citation preview
Mindtalk Technology
Behind the Scenes
Robin Syihab (CTO Mindtalk)MT: @robinTW: @anvie
Apa itu “Scalability”?
“Scalability” bukan
•Performa yang tinggi.
•Mudah untuk menambahkan server.
•High availability.
Skalabilitas adalah
•Kemampuan dalam mengakomodir peningkatan permintaan yang tinggi.
•Bukan hanya masalah server atau teknologi, tetapi juga sumber daya manusia dan proses di dalamnya.
Bagaimana Mindtalk
menghadapi masalah
skalabilitas?
Database
•MongoDB (main database).
•Redis (cache).
•Tokyo Cabinet (url shortener).
Database
MongoDB(master)
MongoDB(replica)
Service
Cache
Write
HAProxyRead
MongoDB(shard)
MongoDB(shard)
MongoDB(shard)
MongoDB(shard)
Web Server
•Nginx
•HAProxy (failover)
Web Service
web service
rest API service
Mobile web service
consumer
HAProxy
Tips Optimasi
•Selalu aktifkan gzip.
•Jangan gunakan domain yang sama untuk web dan serving static files.
•Tuning web level cache.
•nginx workers == jumlah core.
•Jika perlu gunakan varnish.
Elastic Search
•Mudah menambahkan shard, tidak perlu servis tambahan seperti servis kordinator, contoh: Zookeeper.
•Skema dokumen yang flexibel.
•Faceting.
•Pencarian ke banyak index.
•Automatic shard rebalancing.
Development Process
Source Control Management : GIT
Continuous Integration: Buildbot
Code Inspection: Gitlab
Monitoring System: PandoraFMS
http://www.lafermeduweb.net
Deployment System
Makefile increment versionpre-build test
build distribution
backup
staging
updating
Logging
Levels:
DEBUG
INFO
WARNING
ERROR
file A (rotated)
file B (rotated)
file C (rotated)
email to maintainersincludes technical information
Bahasa pemrograman
•Python - Backend services.
•Scala - Stream processing.
•Node.js - Realtime notifier.
•PHP - Front end.
FRONT ENDS
Closure Library
GoAnscdn & AWS
Sock API
web API Cache
Cache
Search Engine
Hadoop
Database
Database
Notifier
Replica
•Authentication•API
Message Queueing
•Mobile Push Notification
•Web Notificationlong polling
Data mining storage
MID ENDS BACK ENDS
•3 instances round-robin by hproxy
•Includes Lucene Engine
•Purpose to:• Generate analytics.• Calculate search relevancy.• Calculate suggestion like PYMK or GYML.
Ndayak 3 EngineNdayak 3 Engine
Geek Wanted
•Java / Scala Programmer.
•NoSQL Database Engineer.
•System Administrator.
kirimkan CV ke:[email protected]
Terima kasih
