Distributed system in Elixir

ELIXIR를 이용한 분산시스템HARU CONFERENCE

OVERVIEW

▸ Erlang/OTP

▸ Elixir

▸ Distributed systems

▸ Wrapping up

ERALNG/OTP

1986

1998

2016

Ericsson 개발

오픈소스화

Erlang OTP 19.x

1986

1998

2016

Ericsson 개발

오픈소스화

Erlang OTP 19.x

1995 Java

SWITCH

SWITCH

http://stackoverflow.com/questions/1636455/where-is-erlang-used-and-why/1637137#1637137

CONCURRENCY.

DISTRIBUTION.

FAULT TOLERANCE.

SERVER

SERVER

http://stackoverflow.com/questions/1636455/where-is-erlang-used-and-why/1637137#1637137

ERLANG/OTP

▸ Functional language

▸ Immutable variables

▸ Strong, dynamic typing

▸ Actor model

▸ OTP

▸ Behavior

FUNCTIONAL LANGUAGE

FUNCTION(X)

A

B

FUNCTIONAL LANGUAGE

FUNCTION(X)

FUNCTION(X)

FUNCTION(X)

ACTOR MODEL

Actor = Lightweight Process + Data + Code

ACTOR

CODE DATA

PROCESS

ACTOR MODEL

ACTOR

CODE DATA

PROCESS

ACTOR

CODE DATA

PROCESS

message

message queuemessage queue

ACTOR MODEL

ACTOR

CODE DATA

PROCESS

message queue

message

ACTOR MODEL

ACTOR

CODE DATA

PROCESS

message

ACTOR

CODE DATA

PROCESS

message

ACTOR MODELLIFE CYCLE

INITIALIZE

TERMINATE

RECEIVE-EVALUATE LOOP

Designing for Scalability with Erlang/OTP

start

stop

ACTOR MODEL

https://viethip.com/category/erlang/

process process process

ERLANG VM

GARBAGE COLLECTOR

ACTOR MODEL

ACTOR MODEL

노동자

관리자

사장

ACTOR MODEL

WORKER

SUPERVISOR

ROOT_SUPERVISOR

OTP-BEHAVIOR

▸ GenServer

▸ Supervisor

▸ GenEvent

▸ GenFsm

▸ GenStatem

OTP-BEHAVIOR

▸ GenServer

▸ Supervisor

▸ GenEvent

▸ GenStatem

GENSERVERGENERIC SERVER

SERVER

CLIENT

CLIENT

CLIENT

http://erlang.org/doc/design_principles/gen_server_concepts.html

requestresponse

SUPERVISOR

ACTOR ACTOR ACTOR

SUPERVISOR


SUPERVISOR

ACTOR ACTOR WORKER

SUPERVISOR

ACTOR ACTOR ACTOR

SUPERVISOR


SUPERVISION TREE

“Erlang was designed for writing concurrent programs that run forever.”

A History of Erlang - Joe Amstrong

ELIXIR

2012

2016

José Valim 개발

Elixir 1.3.x

ELIXIR

▸ Functional language

▸ Immutable variables

▸ Strong, dynamic typing

▸ Actor model

ELIXIR

BEAMOTP

ERLANG ELIXIR

ELIXIR

▸ Support tools

▸ ExUnit - Unit test

▸ Mix - Build tool

▸ Standard library

▸ Metaprogramming

▸ Ecosystem

▸ Hex - Package management

▸ Phoenix - Web framework

DISTRIBUTED SYSTEMS

“A distributed system is one in which the failure of a computer you didn't even know existed can render your own computer unusable”

Leslie Lamport

ERLANG CLUSTER

NODE

A distributed Erlang system consists of a number of Erlang runtime systems communicating with each other. Each such runtime system is called a node.

ERLANG CLUSTER

NODE

NODE

NODE

NODE

ERLANG CLUSTER

NODE NODE

ACTOR ACTORmessage

WHY DISTRIBUTED SYSTEMS?

WHY DISTRIBUTED SYSTEMS?

▸ Enhanced Performance

▸ Higher Availability

PERFORMANCE

▸ Latency

▸ Throughput

▸ Computing power

PERFORMANCE

NODE

CLIENT

PERFORMANCE CLIENT

BUSINESS NODE

LOADBALANCER

ENDPOINT NODE ENDPOINT NODE

BUSINESS NODE

PERFORMANCE

BUSINESS NODE

ENDPOINT NODE

PAYMENT GROUP

BUSINESS NODE BUSINESS NODE

LOGIN GROUP

BUSINESS NODE

PERFORMANCE

BUSINESS NODE

ENDPOINT NODE

PAYMENT GROUP


LOGIN GROUP

BUSINESS NODE

Stateless

PERFORMANCE CLIENT

BUSINESS NODE

LOADBALANCER


BUSINESS NODE

CACHE

STATEFUL #1

PERFORMANCE CLIENT

BUSINESS NODE

LOADBALANCER


BUSINESS NODE

CACHE

STATEFUL #1

PERFORMANCE

BUSINESS NODE +

LOCAL MEMORY

LOADBALANCER


STATEFUL #2

BUSINESS NODE +

LOCAL MEMORY

CLIENT

PERFORMANCE CLIENT#1

BUSINESS NODE +

LOCAL MEMORY

LOADBALANCER


CLIENT#2

STATEFUL #2

BUSINESS NODE +

LOCAL MEMORY

{session : client#1} {session : client#2}

PERFORMANCE

BUSINESS NODE +

LOCAL MEMORY

LOADBALANCER


CLIENT#2STATEFUL #2

BUSINESS NODE +

LOCAL MEMORY

{session : client#1} {session : client#2}

PERFORMANCE

RIAK_CORE

STATEFUL #2

PERFORMANCESTATEFUL #2

PERFORMANCESTATEFUL #2

NODE

VNODE1 VNODE2

NODE

VNODE3 VNODE4

NODE

VNODE5 VNODE6

…

hash(session_id) = 1

https://www.youtube.com/watch?v=sYYOLaJ-VDQ

PERFORMANCEDesigning for Scalability with Erlang/OTP

PERFORMANCE

SYNCHRONOUS CALLS

PERFORMANCESYNCHRONOUS CALLS

ACTOR A

ACTOR B

ACTOR C

ACTOR D

BC

A

PERFORMANCE

EXTERNAL SERVICE

PERFORMANCE CLIENT

BUSINESS NODE

LOADBALANCER


BUSINESS NODE EXTERNAL SERVICEEXTERNAL SERVICE

PERFORMANCE

LOAD REGULATION AND BACKPRESSURE

PERFORMANCE

ENDPOINT NODE

BUSINESS NODE

EXTERNAL SERVICEEXTERNAL SERVICE

BACKPRESSURE

LOAD REGULATION

ENDPOINT NODE

BUSINESS NODE

BACKPRESSURE

LOAD REGULATION

CLIENTS

AVAILABILITY

▸ Fault Tolerance

▸ Resilience

▸ Reliability

AVAILABILITYFAULT TOLERANCE

CLIENT

BUSINESS NODE

ENDPOINT NODE

CLIENT

BUSINESS NODE

ENDPOINT NODE

error

error


NODE NODE

ACTOR ACTORasynchronous request


NODE NODE

ACTOR ACTORtimeout

BUSINESS NODE

ENDPOINT NODE

PAYMENT GROUP


LOGIN GROUP

BUSINESS NODE

AVAILABILITYRESILIENCE - NODE

BUSINESS NODE

ENDPOINT NODE

PAYMENT GROUP


LOGIN GROUP

BUSINESS NODE


PAYMENT GROUP

ENDPOINT NODE


LOGIN GROUP

BUSINESS NODE


BUSINESS NODE

ENDPOINT NODE

PAYMENT GROUP


LOGIN GROUP

BUSINESS NODE


ACTOR ACTOR ACTOR

SUPERVISOR

AVAILABILITYRESILIENCE - ACTOR

ACTOR ACTOR ACTOR

SUPERVISOR


ACTOR ACTOR ACTOR

SUPERVISOR


ACTOR

AVAILABILITYRELIABILITY

CLIENT

ENDPOINT NODE

BUSINESS NODEBUSINESS NODE


CLIENT

ENDPOINT NODE



CLIENT

ENDPOINT NODE


WRAPPING UP

NO SILVER BULLET

Designing for Scalability with Erlang/OTP

Programming Elixir 1.3

Erlang/Elixir Korea User Grouphttps://www.facebook.com/groups/elixir.korea/

REFERENCES▸ Books

▸ Designing for Scalability with Erlang/OTP

▸ Programming Elixir 1.3 Reliability

▸ The_Little_Elixir_&_OTP_Guidebook

▸ Videos

▸ Experimenting with Superpowered Web Services: Phoenix on Riak_Core https://www.youtube.com/watch?v=sYYOLaJ-VDQ

▸ Erlang Factory SF 2016 - Concurrency + Distribution = Scalability + Availability, a Journey Architecting Erlang Systems https://www.youtube.com/watch?v=_IZMQMuphfo

https://www.youtube.com/watch?v=sYYOLaJ-VDQ

https://www.youtube.com/watch?v=_IZMQMuphfo

THANK YOU…

Technology

Distributed system in Elixir