Elasticsearch And Ruby [RuPy2012]

  • View

  • Download

Embed Size (px)



Text of Elasticsearch And Ruby [RuPy2012]

  • 1. ElasticsearchAnd RubyKarel Minak

2. http://karmi.czElasticsearch and Ruby 3. {elasticsearch in a nutshell}Built on top of Apache LuceneSearching and analyzing big dataScalabilityREST API, JSON DSLGreat fit for dynamic languages and web-oriented workflows / architectureshttp://www.elasticsearch.org Elasticsearch and Ruby 4. { }Elasticsearch and Ruby 5. { }It all started in this gist (< 200 LOC)Elasticsearch and Ruby 6. { }Elasticsearch and Ruby 7. Example class Results include Enumerable attr_reader :query, :curl, :time, :total, :results, :facetsdef initialize(search)response = JSON.parse( Slingshot.http.post("http://localhost:9200/#{search.indices}/_search", search.to_json) )@query = search.to_json@curl= %Q|curl -X POST "http://localhost:9200/#{search.indices}/_search?pretty" -d #{@query}|@time= response[took]@total = response[hits][total]@results = response[hits][hits]@facets = response[facets]end def each(&block) @results.each(&block) end end Elasticsearch plays nicely with RubyElasticsearch and Ruby 8. elasticsearchs Query DSLcurl -X POST "http://localhost:9200/articles/_search?pretty=true" -d { "query" : { "filtered" : { "filter" : { "range" : { "date" : { "from" : "2012-01-01", "to" : "2012-12-31" } } }, "query" : { "bool" : { "must" : { "terms" : { "tags" : [ "ruby", "python" ] } }, "must" : { "match" : { "title" : { "query" : "conference", "boost" : 10.0 } } } } } } }} 9. Example Tire.search(articles) do query do boolean do must { terms :tags, [ruby, python] } must { string published_on:[2011-01-01 TO 2011-01-02] } end end end Elasticsearch and Ruby 10. Example tags_query = lambda do |boolean| boolean.must { terms :tags, [ruby, python] } end published_on_query = lambda do |boolean| boolean.must { string published_on:[2011-01-01 TO 2011-01-02] } end Tire.search(articles) do query { boolean &tags_query } end Tire.search(articles) do query do boolean &tags_query boolean &published_on_query end end Elasticsearch and Ruby 11. Example search = Tire.search articles do query do string title:T* end filter :terms, tags: [ruby] facet tags, terms: tags sort { by :title, desc } endsearch = Tire::Search::Search.new(articles)search.query { string(title:T*) }search.filter :terms, :tags => [ruby]search.facet(tags) { terms :tags }search.sort { by :title, desc } Elasticsearch and Ruby 12. TEH PROBLEM Designing the Tire library as domain-specific language, from the higher level, and consequently doing a lot of mistakes in the lower levels. Class level settings (Tire.configure); cannot connect to two elasticsearch clusters in one codebase Inconsistent access (methods vs Hashes) Not enough abstraction and separation of concerns Elasticsearch and Ruby 13. Blocks with arguments(alternative DSL syntax) Tire.search do query do text :name, params[:q] end endTire.search do |search|search.query do |query|query.text :name, params[:q]endendElasticsearch and Ruby 14. The Git(Hub) (r)evolution Lots of contributions... but less feedback Many contributions focus on specific use case Many contributions dont take the bigger picture and codebase conventions into account Almost every patch needs to be processed, polished, amended Maintainer: lots of curation, less development even on this small scale (2K LOC, 7K LOT) Contributors very eager to code, but a bit afraid to talk 15. Tires Ruby on Rails integration$ rails new myapp -m "https://raw.github.com/karmi/tire/master/examples/rails-application-template.rb" Generate a fully working Rails application with a single command Downloads elasticsearch if not running, creates the application, commitsevery step, seeds the example data, launches the application on a free port, Tire::Results::Item fully compatible with Rails view / URL helpers Any ActiveModel compatible OxM supported Rake task for importing data (using pagination libraries)Elasticsearch and Ruby 16. Rails integration baked in No proper separation of concerns / layers People expect everything to be as easy as that Tire::Results::Item baked in, not opt-in, masquerades as models People consider ActiveRecord the only OxM in the worldElasticsearch and Ruby 17. Persistence extensionRails extensionsActiveRecord extensionsActiveModel integrationThe Ruby DSLBase library (HTTP, JSON, API) 18. https://rubygems.orghttps://github.com/rubygems/rubygems.org/pull/455 19. Searchclass Rubygem < ActiveRecord::Base# ...def self.search(query)conditions =