Click here to load reader

Introduction to Elasticsearch

  • View
    1.018

  • Download
    1

Embed Size (px)

Text of Introduction to Elasticsearch

  • @somkiat

    SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

  • About me

    SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    somkiat.ccSPRINT3R

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    IntroductionSpeed by example

    Search Aggregation

    Agenda

    Operating System and Hardware

  • SELECT

    SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    FROM TABLE

    WHERE TEXT LIKE %SHIT%

    Search system

  • SELECT

    SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    FROM many Table

    GROUP BY some column

    Report/analytic system

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    Saaaaaaaadddddddddd

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    First version 0.4 in Feb 2010Rewrite from Compass projectAdd scalability

    Created by Shay Banon

    Current version 1.5.1

    History

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    an open source, distributed, scalable,

    highly availability, document-oriented, RESTful

    Elasticsearch

    full text search engine

    with near real-time search and analytics

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    Elasticsearch

    an open source, distributed, scalable, highly availability, document-oriented, RESTful

    full text search enginewith near real-time search and analytics

    Apache 2.0 Licence

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    Elasticsearch

    an open source, distributed, scalable, highly availability, document-oriented, RESTful

    full text search enginewith near real-time search and analytics

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    Elasticsearch

    an open source, distributed, scalable, highly availability, document-oriented, RESTful

    full text search enginewith near real-time search and analytics

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    Elasticsearch

    an open source, distributed, scalable, highly availability, document-oriented, RESTful

    full text search enginewith near real-time search and analytics

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    Elasticsearch

    { "id" : 1, "name" : "somkiat", "family_name" : "puisungnoen", "website" : "somkiat.cc" }

    an open source, distributed, scalable, highly availability, document-oriented, RESTful

    full text search enginewith near real-time search and analytics

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    Elasticsearch

    an open source, distributed, scalable, highly availability, document-oriented, RESTful

    full text search enginewith near real-time search and analytics

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    Elasticsearch

    an open source, distributed, scalable, highly availability, document-oriented, RESTful

    full text search enginewith near real-time search and analytics

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    Elasticsearch

    an open source, distributed, scalable, highly availability, document-oriented, RESTful

    full text search enginewith near real-time search and analytics

  • realtime, search and analytics engine

    SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    Lucene

    DistributedScalability

    open-sourcedocumentation

    JSON APIRESTFul

    document store

    High-Availability

    JAVA

    Plug-ins

    Elasticsearch

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    ?

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    Search repositories, users, issues, pull request Search sourcecode 130 Track alerts, events logs

    ?

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    Full text seach + geolocation feature More-like-this

    ?

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    Java PHP Ruby Python JavaScript

    Go Scala .Net Clojure Erlang

    NodeJS R

    http://www.elasticsearch.org/guide/en/elasticsearch/client/community/current/clients.html

    Clients

  • Elasticsearch

    SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    Elasticsearch

    $ wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.5.1.zip

    $ unzip elasticsearch-1.5.1.zip $ cd elasticsearch-1.5.1

    $ bin/elasticsearch

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    Elasticsearch

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    Elasticsearch

  • Scaling

    SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    Cluster :: node

    node

    cluster

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    Cluster :: node

    node

    cluster

    node

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    Cluster :: node

    node

    cluster

    node

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    Cluster :: node

    node

    cluster

    node node

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    Node :: 4

    master data

    search

    master search

    data

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    Shard :: unit of scale

    0 12 3

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    Shard :: unit of scale

    0 1 2 3

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    Replication

    0 1 2 33 0 1 2

  • Search

    SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    RDBMS vs Elasticsearch

    D ATA B A S E TA B L E R O W C O L U M N

    I N D E X T Y P E D O C U M E N T F I E L D

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    CRUD

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    CRUD :: Create

    PUT // { your data .. }

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    CRUD :: Create

    PUT shop/book/1 { "name" : "Elasticsearch: The Definitive Guide", "authors" : [ "Clinton Gormley", "Zachary Tong" ], "pages": 724, "published_date": "2015/01/31" }

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    CRUD :: Read

    GET shop/book/1

    GET shop/book/_search

    GET shop/book/_search?q=elasticsearch

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    CRUD :: Delete

    DELETE shop/book/1

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    Searching with DSL

    GET shop/book/_search { "query": { "filtered": { "query": { "match": { "name": "elasticsearch"}}, "filter": { "range": { "published_date": { "gte": "now-1y" }} } } } }

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    Search response in JSON

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    Search from all fields

    GET shop/book/_search { "query": { "filtered": { "query": { "match": { "_all": "elasticsearch"}}, "filter": { "range": { "published_date": { "gte": "now-1y" }} } } } }

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    Reduce search response in JSON

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    Reduce search response in JSON

    GET shop/book/_search { "_source": ["name", "pages"], }

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    Query vs Filter

  • Aggregations

    SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    High level

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    Aggregations

    Buckets and Metrics

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

    Aggregations

    Documents

  • SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Al

Search related