20151022 elasticsearch 적용및활용_송준이_sds발표용

PowerPoint

2015.10.22([email protected])

Developer Guide

?

?

about elasticsearch

inside a cluster

inside a shard

title: 1lyrics: .....titlelyrics12

table in RDBMS row full scan=> 1Mtermdocs1, 7, 3, 23, 54,

inverted index in search engine row full scan=> 1

melon-helloEshttp://socurites.com:8088/melon-helloes-0.0.1-SNAPSHOT/overview/intro

elasticsearch-headhttp://socurites.com:9200/_plugin/head/

Marvel Sensehttp://socurites.com:9200/_plugin/marvel/sense/index.html

about elasticsearch

(distributed document-oriented) (data store) scale outPB document(serialized JSON object) data structurepartial document update document (immutable)update = replacement(internally) (search engine)(lucene) indexing (real-time analytic platform)aggregation approximate aggregation = (big data + real-time analysis) precision

storingindexingsearchinganalyzingfilteringorderingaggregation

life inside a cluster

node & cluster0 index > 1 node > 1 cluster

node: elasticsearch instancemaster node index / node / master node document search master node bottleneck cluster: cluster.name node

) cluster / node / indexhttps://github.com/socurites/elasticsearch-foot

index & shard3 primary shard > 1 index > 1 node 1 cluster

index: shard shard: index index indexing document document shard scale out primary shard document primary shard primary shard index (default: 5)replica shardprimary shrad recovery / searching concurrent read

1 replica3 primary shard 1 replica > 1 index > 2 node 1 cluster

scale out shard reallocation3 primary shard 1 replica > 1 index > 3 node 1 cluster

shard , computing power

2 replica3 primary shard 2 replica > 1 index > 3 node 1 cluster

recovery on failure3 primary shard 2 replica > 1 index > 2 node 1 clustermaster node 1 shutdown ,

primary node selectionnode 2 primary node recovering primary shardnode 1 primary shard 1, 2 replica node primary shard

inside a shard

flush, refresh, optimizeelasticsearchsearch near real-time: indexing document 1s search CRUD real-timedata persistence delete operation disk

Why?refreshflushoptimize

dynamically updatable indicesindex immutable , index index inverted index document index search , index segment: index

index = segements + commit pointcommit point= segment indexshardsegment

http://socurites.com/big-data/elasticsearch/elasticsearchs-index-structrue-segment-merging