Download pdf - Introduction to Elasticsearch

@somkiat

SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

About me


สมเกยรต ปยสงเนน

สยามานาญกจ

somkiat.ccSPRINT3R


Introduction

Speed by example• Search• Aggregation

Agenda

• Operating System and Hardware

SELECT


FROM TABLE

WHERE TEXT LIKE ‘%SHIT%’

Search system

SELECT


FROM many Table

GROUP BY some column

Report/analytic system


Saaaaaaaadddddddddd


First version 0.4 in Feb 2010Rewrite from Compass projectAdd scalability

Created by Shay Banon

Current version 1.5.1

History


an open source, distributed, scalable,

highly availability, document-oriented, RESTful

Elasticsearch

full text search engine

with near real-time search and analytics


Elasticsearch

an open source, distributed, scalable, highly availability, document-oriented, RESTful

full text search enginewith near real-time search and analytics

Apache 2.0 Licence


Elasticsearch




Elasticsearch




Elasticsearch




Elasticsearch

{ "id" : 1, "name" : "somkiat", "family_name" : "puisungnoen", "website" : "somkiat.cc" }




Elasticsearch




Elasticsearch




Elasticsearch



realtime, search and analytics engine


Lucene

DistributedScalability

open-sourcedocumentation

JSON APIRESTFul

document store

High-Availability

JAVA

Plug-ins

Elasticsearch


ใครใบาง ?


• Search repositories, users, issues, pull request• Search sourcecode 130 พนลานบรรทด• Track alerts, events และ logs



• ใ Full text seach + geolocation• ใ feature More-like-this ในการหาคาถาม และ คาตอบ



• Java• PHP• Ruby• Python• JavaScript

• Go• Scala• .Net• Clojure• Erlang

• NodeJS • R

http://www.elasticsearch.org/guide/en/elasticsearch/client/community/current/clients.html

Clients

http://www.elasticsearch.org/guide/en/elasticsearch/client/community/current/clients.html

สวสด Elasticsearch




$ wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.5.1.zip

$ unzip elasticsearch-1.5.1.zip $ cd elasticsearch-1.5.1

$ bin/elasticsearch





Scaling



Cluster :: กลมของ node

node

cluster



node

cluster

node



node

cluster

node



node

cluster

node node


Node :: ม 4 ชนด

master data

search

master search

data


Shard :: unit of scale

0 12 3


Shard :: unit of scale

0 1 2 3


Replication

0 1 2 33 0 1 2

Search



RDBMS vs Elasticsearch

D ATA B A S E TA B L E R O W C O L U M N

I N D E X T Y P E D O C U M E N T F I E L D


CRUD


CRUD :: Create

PUT <index name>/<type>/<id> { … your data .. }


CRUD :: Create

PUT shop/book/1 { "name" : "Elasticsearch: The Definitive Guide", "authors" : [ "Clinton Gormley", "Zachary Tong" ], "pages": 724, "published_date": "2015/01/31" }


CRUD :: Read

GET shop/book/1

GET shop/book/_search

GET shop/book/_search?q=elasticsearch


CRUD :: Delete

DELETE shop/book/1


Searching with DSL

GET shop/book/_search { "query": { "filtered": { "query": { "match": { "name": "elasticsearch"}}, "filter": { "range": { "published_date": { "gte": "now-1y" }} } } } }


Search response in JSON


Search from all fields

GET shop/book/_search { "query": { "filtered": {

"query": { "match": { "_all": "elasticsearch"}}, "filter": { "range": { "published_date": { "gte": "now-1y" }} } } } }


Reduce search response in JSON


Reduce search response in JSON

GET shop/book/_search { "_source": ["name", "pages"], … }


Query vs Filter

Aggregations



High level


Aggregations

Buckets and Metrics


Aggregations

Documents


Aggregations

Query/Filter

Documents


Aggregations

Buckets

Query/Filter

Documents


Aggregations

Documents

Query/Filter

Buckets

Metrics 2 1 4


Aggregations

ผแตงแตละคนมหนงอกเลม ?


Aggregations

GET shop/book/_search?search_type=count { "aggs": { "book_by_author": { "terms": { "field": "authors" } } } }


Results :: ปญหาคออะไร ?


Results :: ปญหาคออะไร ?


ปญหาจาก Inverted index

Term Document IDclinton 1

gormley 1

tong 1

zachary 1


งทตองการจาก Inverted index

Term Document IDClinton Gormley 1

Zachary Tong 1


กาหนด mapping

PUT shop { "mappings": { "book": { "properties": { "authors": { "index": "not_analyzed", "type": "string" } } } } }


Results

Operating System & Hardware



• CPU

• Memory

Elasticsearch use

• I/O

• Network

Indexing, searching

Indexing, searching, merging

Aggregation, indices

Relocation, snapshot & restore

Summary


Summary


Speed is key …

Summary


Search Tradeoff

Summary


Thank you and Questions
