Upload
ontico
View
1.591
Download
4
Embed Size (px)
Citation preview
Tarantool as amicroservicesplatformAnton ReznikovVladimir Perepelitsa
100k+ RPSper cpu core
$1 000 000+comparing to big cluster
What else…
SQL?
Memcached?
Queues?
$$$
Roadmap
RoadmapKV with expiration
RoadmapKV with expiration
Smart caching proxy
RoadmapKV with expiration
Smart caching proxy
Complex queues
RoadmapKV with expiration
Smart caching proxy
Complex queues
Application server
RoadmapKV with expiration
Smart caching proxy
Complex queues
Application server
Custom cluster solutions
Get yourdata in RAM
Computeclose to data
Enjoy theperformance
Average performanceOperation Req / s / CPUSelect: 150kInsert: 75kUpdate: 75kDelete: 90kLuaCall: 75k
Indexes:HASH
Indexes:HASH
TREE: SG, AVL (1.5) | B+* (1.6)
Indexes:HASH
TREE: SG, AVL (1.5) | B+* (1.6)
RTREE (1.6)
Indexes:HASH
TREE: SG, AVL (1.5) | B+* (1.6)
RTREE (1.6)
BITMAP
Indexes:HASH
TREE: SG, AVL (1.5) | B+* (1.6)
RTREE (1.6)
BITMAP
Composite
Indexes:HASH
TREE: SG, AVL (1.5) | B+* (1.6)
RTREE (1.6)
BITMAP
Composite
Multiple
Lua
Luajit!
Luajit!fibers (green threads)Cooperative multitasking
Luajit!fibers (green threads)Cooperative multitasking
channels (fiber communicaion)Synchronization primitives
Luajit!fibers (green threads)Cooperative multitasking
channels (fiber communicaion)Synchronization primitives
sockets (client & server)Pure raw, like in C and with helpers
Luajit!fibers (green threads)Cooperative multitasking
channels (fiber communicaion)Synchronization primitives
sockets (client & server)Pure raw, like in C and with helpers
FFI (foreign function interface)Ability to call C/C++ (dlopen)
8 lines
Lua: join by handfunction custom_join( key ) local result = {}
for tuple in space:iterator('EQ', key) do local joined = other:select(0,tuple[1])
table.insert(result,{ tuple, joined }) end
return resultend
Access tokensProvides access by URL tospecific feature withoutadditional authorization
Durable
Temporary
Consistent
Token
Payload
Token
Payload
index
Lua
TokenExpire time
Payload
fiber
index
Lua
TokenExpire time
User id
Payload
index
fiber fiber
Classic SQL AppWeb application with 20krequests for select and 2krequests for update
Have memcached
Have SQL replica
get
setApp
missMemcache
SQL
App Tarantool
SQL
App Tarantool
SQL
App Tarantool
SQL
App Tarantool
replicaSQLSlave
chanscache
query 1 query SQL
query 2
reply 2
chan
SQL responsereply 1
query 3reply 3
20 lines!
Aggregation: examplefunction select(key) local tuple = cache:select {key} if not tuple then if locks[key] then local ch = fiber.channel(1) locks[key][ch] = ch return ch:get() else locks[key] = {} local ret = sql:execute('select * from ...') for ch in pairs(locks[key]) do ch:put(ret) end locks[key] = nil return ret end else return tuple endend
OAuth tokensStorage for 10M OAuthaccess+refresh tokens withdeadline refreshing
Keep minimal expired tokens
Balance load over time
App WorkerTokens
queueSched
App WorkerTokens
queue
G
MS
Y
Я
Sched
App WorkerTokens
queues
G
MS
Y
Я
Sched
Queue priority
to refresh no need to refreshexpired
60sfirst
order
old agesecondorder 5 min
thirdorder
App WorkerTokens
G
MS
Y
Я
Subscriptionverification
Mobile application providesubscription id, that must beverified in store
Don't duplicate requests
Reply as soon as possible
Provide service if client gone
App
Worker
wait
create task
wait
have task
x3done task
id
Apple
Database applicationReplicated database with a lotof business logic inside
Multiple APIs
Failsafe deploy
Monitoring
Informative logs
App
API
v1
App
Lua
API
v1
App
App
v2
Lua
API
v1
App
App
v2
Lua
logs
Mon
Stat
world
fiber
Sharding isunavoidable
Data does not fit in RAM of asingle instance or many CPUcores required
Database as a service
Consistency checking
#1
#2
#3
App Sh prxf(k)
App
App Sh prxf(k)
Sh prxf(k)
#1
#2
#3
App Sh prxf1(k)
App
App Sh prxf(k)
Sh prxf(k)
#1
#2??
#3
App Sh prxf(k)
App
App Sh prxf(k)
Sh prxf(k)
#1f(k)
#2f(k)
#3f(k)
App
App
App
Access tokens (v2)Provides access by URL tospecific feature withoutadditional authorization
High availability
Allow loss of majority
M
M
M
APP #1
APP #2
M
M
M
APP #1
APP #2
M
M
M
APP #1
APP #2
Get yourdata in RAM
Computeclose to data
Enjoy theperformance
tarantool.orgDocumentation:
try.tarantool.orgTry it online:
github.com/tarantoolExplore it:
github.com/mons/tnt-luasLuas:
Questions?