Upload
poga-po
View
191
Download
0
Embed Size (px)
Citation preview
萬事萬物皆是 LOG
Poga @ COSCUP 2016.08.20
LOGIf you want maintainable, understandable software system
什麼是 LOG
LOG 的性質
● 按照時間出現
● 出現過的就不會改變(append-only)
● 目的:記錄(某個時間)發生了(某件事)○ 常常用來作為 debug 推理的依據
Things based on LOG
● Database● Version Control● Distributed System and Distributed Consensus● Synchronization● Replication● Messaging● UI Framework● ...And almost EVERYTHING
聽完就理解了軟體架構的真理(誤)
POGASoftware Developer, g0v.tw, KKTIX(前)
資料庫
ID Username Balance
231 poga 100
7212 moon 12314
834 et 927
... ... ...Database
● 大量讀寫資料,長期儲存
● 確保資料正確性(relation, validation, schema, ACID)
● 支援多種高效查詢(SQL, index)
如果你從沒用過資料庫...
寫入資料時當機?
如何避免寫入資料時當機導致資料消失?
寫入:ID Username Balance
231 poga 100
...2016.08.20 13:40:05.0012621 Write {ID:231, Username:poga, Balance: 100}...
真正寫入: Validation, Update Index… etc.
...2016.08.20 13:40:05.0012621 Write {ID:231, Username:poga, Balance: 100}...
時間 發生了什麼事
WRITE-AHEAD LOG
LOG
● Fast and simple
RECAP
資料庫
ID Username Balance
231 poga 100
7212 moon 12314
834 et 927
... ... ...Database
● 單一 Database 擴充性有其極限
● 多台 Database,分散負載○ 需要確保多台 Database 之間資料同步
Replication
確保多台資料庫間資料同步?
寫入一台資料庫
寫入:ID Username Balance
231 poga 100
...2016.08.20 13:40:05.0012621 Write {ID:231, Username:poga, Balance: 100}...
真正寫入: Validation, Update Index… etc.
一次寫入多台資料庫
寫入:ID Username Balance
231 poga 100
...2016.08.20 13:40:05.0012621 Write {ID:231, Username:poga, Balance: 100}...
真正寫入: Validation, Update Index… etc.
...2016.08.20 13:40:05.0012621 Write {ID:231, Username:poga, Balance: 100}...
真正寫入: Validation, Update Index… etc.
DB1 DB2
LOG-SHIPPING
LOG
● Fast and simple● Can be used to share “current state”
RECAP
微服務Microservice
Monolith v.s. Microservice
Monolith
● 所有邏輯集中在單一系統中,共
享儲存運算資源
● 容易開發
● 不易擴充
Microservice
● 將邏輯按照 Domain 切割,有
獨立的儲存與運算資源
● 容易擴充,團隊間分責明確
● 門檻較高:不易維護,需要進階
系統維護能力
微服務間怎麼互相溝通?
Event Stream
EVENT STREAM = LOG
LOG
● Fast and simple● Can be used to share “current state”● LOG are easy to parse/understand, Universal Interface
RECAP
前端架構Flux/Redux
● Many, many states
● Constantly changing state○ User Interaction, Animation, Network
● Hard to reproduce bug and debug
UI State
前端介面狀態多變,如何維護?
Flux/Redux
● Unidirectional Data Flow○ Append only○ Handle action one by one, reasonable
● Immutable State○ Produce new state based on previous state and action
● Everything else can be stateless(pure)
UNIDIRECTIONAL = LOG
LOG
● Fast and simple● Can be used to share “current state”● Text are easy to parse/understand, Universal Interface● Append-only, easy to reason, everything else can be
stateless
RECAP
區塊鏈Blockchain
● 2016.08.20 13:31:23, A 給 B 100 bitcoin
● State: A 跟 B 身上剩下多少 bitcoin
● Double Spend: ○ A 把 bitcoin 交給 B 後,不能再把同樣的 bitcoin 交給 C
Transaction
如何讓全世界都認知交易的成立?a.k.a. Avoid Double-Spending
● 如果○ 所有參與者都知道/可以驗證所有人的交易記錄
○ 既有的交易紀錄無法被竄改
○ 便沒有偽造空間
● Blockchain○ 將所有的交易記錄用 block 封裝,每個 block 替之前的交易記錄加密驗證
○ 想要偽造一筆交易記錄 = 必須偽造整個 blockchain 上的交易記錄
○ 所有人都基於相同的 blockchain 驗證交易
Distributed Consensus
LOG AGREEMENT = CONSENSUSRaft: A Consensus Algorithm for Replicated Logs
Apache Mesos - The Mesos Replicated Log (PAXOS)
LOG
● Fast and simple● Can be used to share “current state”● Text are easy to parse/understand, Universal Interface● Append-only, Immutable, easy to reason, everything else
can be stateless● Replicate log = consensus
RECAP
大數據
Data Pipeline
● > PB● Data coming from everywhere
○ User Input○ Business Data○ Tracking○ External Data
● ETL(Extract-Transform-Load)● Analysis, Machine Learning… etc
BIG Data
如何處理雜亂、無序、大量的資料?
Data Pipeline
APPEND ONLY LOG = HIGH SCALABILITY
LOG
● Fast and simple● Can be used to share “current state”● Text are easy to parse/understand, Universal Interface● Append-only, Immutable, easy to reason, everything else
can be stateless● Replicate log = consensus● High Scalability
RECAP
LOG = DETERMINISM
相同的過程 = 相同的結果DETERMINISM
Make your system deterministic
● 容易理解
● 容易DEBUG● 容易同步
● 容易儲存
● 容易擴展
LOG = DETERMINISM = 穩定的系統
設計系統架構時,先試著用 LOG 表示你的系統
Thanks!