Click here to load reader
View
472
Download
6
Embed Size (px)
AngularJS + Asp.Net Web Api, Signalr, EF6, Redis + Elasticsearch(6/6) - Web : ([email protected])1. AngularJS 2. AngularJS + HighChart3. AngularJS + Asp.Net WebApi 2 +SignalR4. AngularJS + Asp.Net WebApi 2+ SignalR + Entity Framework 6 (ORM)5. AngularJS + Asp.Net WebApi 2+ SignalR +Entity Framework 6 (ORM) + Redis (Nosql)6. AngularJS + Asp.Net WebApi 2+ SignalR +Entity Framework 6 (ORM) + Redis (Nosql) + Elasticsearch ()1Document, Source code & Training Video (6/6)https://github.com/erhwenkuo/PracticalCoding
2Previous Training Session Document, Source code & Training Video (5/6)https://www.youtube.com/watch?v=Xfu4EVBdBKo
http://www.slideshare.net/erhwenkuo/05-integrate-redisSession, , YoubeSlideshare3AgendaElasticsearch IntroductionElasticsearch WorkshopElasticsearch in action using Stackoverflow DatadumpDeveloping Angularjs with ElasticsearchHighchart , AngularJS ,Web API2 , SignalR2 , EF6 , Redis + Elasticsearch Integration
4Elasticsearch Introduction
5Elasticsearch WebsiteHttp://www.elasticsearch.com/
6Whos using elasticsearch?GitHubGitHub uses Elasticsearchs robust sharding and advanced queries to serve up search across data in 4 million users code repositories.GitHub uses Elasticsearchs routing parameter and flexible sharding schemes to perform searches within a single repository on a single shard, doubling the speed at which results are served.GitHub uses Elasticsearchs histogram facet queries, as well as other Elasticsearch analytic queries, to monitor their internal infrastructure for abuse, bugs and more.
7Whos using elasticsearch?WikipediaElasticsearchs reference manual and contribution documentation promised an easy start and pleasant time getting changes upstream when needed to.Elasticsearchs super expressive search API lets Wikimedia search any way needed and gives the company confidence that it can be expanded, including via expressive ad-hoc queries.Elasticsearchs index maintenance API lets Wikimedia maintain the index right from its MediaWiki extension
8Whos using elasticsearch?
3 machines doing search with ElasticSearch
stackoverflow
9ElasticWho?ElasticSearch is a flexible and powerful open source, distributed real-time search and analytics engineFeatures:Real time analyticsDistributedHigh availabilityMulti tenant architectureFull text indexDocument orientedSchema freeRESTful APIPer-operation persistence
10Elasticsearch Workshop
11Download & StartDownload Elasticsearch (Current version: 1.4.2)http://www.elasticsearch.com/downloadsUnzip & Modify elasticsearch.ymlcluster.name: {your_searchcluster_name}node.name: {your_cluster_node_name}http.cors.enabled: trueUse command console to run:bin/elasticsearch on *nixbin/elasticsearch.bat on Windows
123
12Install Elasticsearch Plugins (elasticsearch-head)A web front end for an Elasticsearch clusterhttps://github.com/mobz/elasticsearch-head
You need to have internet access, otherwise it would fail!!1Restart elasticsearch and key in below url in browser:http://localhost:9200/_plugin/head/
213RESTful interfaceElasticsearch default use port 9200 for Http Restful interfaceLets check if Elasticsearch is alive!!
14Elasticsearch TERMs
15
Create IndexElasticsearch Index is similar like Database in relational DBFor example, create a index named stackoverflow
Default setting in Elasticsearch:Each Index will split to 5 shards and has 1 replication
16Create Another Elasticsearch NodeCopy elasticsearch-1.4.2 folder to elasticsearch-1.4.2-Node2Modify elasticsearch-1.4.2-Node2/config/elasticsearch.ymlcluster.name: {your_searchcluster_name}node.name: {your_cluster_node_name-Node2}http.cors.enabled: truetransport.tcp.port: 9301http.port: 9201If we set different Elasticsearch Node on the sameMachine, then we need to assisgn different communication ports.
17
Start All Elasticsearch NodesUse command console to start two Elasticsearch NodesNow we a Elasticsearch Cluster!So so so easy~!18Delete Index
The Index is deleted!!19
Index a Document under a specific TypeType is similar to Table in RelationDB
Document.Id is the unique Id to identify each documentThe content of a Document20
Index a DocumentUnber Browser tab, we can search document21
Get a DocumentType is similar to Table in RelationDB
Document.Id is the unique Id to identify each documentThe content of a Document return by Elasticsearch
22
Update a DocumentType is similar to Table in RelationDB
Document.Id is the unique Id to identify each documentThe version no. of document is incremental if it got updated!!23Searching Document
Control Searching Scope base on URLElasticsearch has very rich Search/Query DSL for document searching. Check below URL for details:http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl.html24
Searching ResultThe documents which hits the searching query will be located under hits/hitsHow long it takes to calculate the result (in milli-seconds)25
Delete a DocumentType is similar to Table in RelationDB
Document.Id is the unique Id to identify each document26Elasticsearch in action using Stackoverflow Datadump
27
Environment Setup Elasticsearch C# ClientUse NuGet to search NEST (C# elasticsearch client)Click Install
28NEST (Elasticsearch C# client)NEST provides very friendly C# interfaces to interact with Elasticseach, also there are many sample codes in its web sitehttp://nest.azurewebsites.net/
29Stackoverflow DatadumpStackoverflow periodically dump their data for public study usehttps://archive.org/details/stackexchangeFor demonstration purpose, we pick a smaller datasetapple.stackexchange.com.7z (73.7MB)Badges.xmlCommenets.xmlPostHistory.xmlPostLinks.xmlPosts.xml (116,071 records)Tags.xmlUsers.xmlVotes.xml
30Session_06_DataloadToElasticsearchA new C# Console program project (Session_06_DataloadToElasticsearch) is created to parsing data dump xml file and import into ElasticsarchOpen Program.cs file and modify below:
Change the Uri of your elasticsearch cluster IP & portChange the xmlDataFile path to the location of data dump fileThis is the Index need to be created before running this program31Execute Session_06_DataloadToElasticsearchProgram.cs
123It spends 67 seconds to index 116071 records32Exploer Data via Brower
33The _search endpoingTo search with ElasticSearch we use the _search endpointWe make http requests to an URL following this pattern: (index & type are both optional)//_searchFor example:Search across all indexes and all typeshttp://localhost:9200/_searchSearch across all types in the stackoverflow indexhttp://localhost:9200/stackoverflow/_searchSearch explicitly for documents of type post within the stackoverflow indexhttp://localhost:9200/stackoverflow/post/_search
34Search request body and ElasticSearch's query DSL elasticsearch provides a full Query DSL based on JSON to define querieshttp://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl.htmlThink of it like ElasticSearch's equivalent of SQL for a relational databaseQuery DSL contains:QueryFilter*** Filter are very handy since they perform an order of magnitude better than plain query since no scoring is performed and they are automatically cached
35Basic free text search The query DSL features a long list of different types of queries that we can useFor "ordinary" free text search we'll most likely want to use one called "query string query".
36Elasticsearch Query DSL - FilterAs a general rule, filters should be used instead of queryies:for binary yes/no searchesfor queries on exact valuesFilters can be caching. Caching the result of a filter does not require a lot of memory, and will cause other queries executing against the same filter (same parameters) to be blazingly fast
37Filter without Query
38Developing Angularjs with Elasticsearch
39Elasticsearch Javascript Client LibraryGo to Elasticsearch web site & get javascript clientelasticsearch-js-2.4.3.ziphttp://www.elasticsearch.org/guide/en/elasticsearch/client/javascript-api/current/browser-builds.htmlUnzip and import to PracticalCoding.Web Project under /Scripts/elasticsearch folder
HightchartHighsoft AS. 2014, 14, 8.
Hightcharts, javascriptsinclude40Setup Fulltext-Search SPA SkeltonCreate 11_AngularWithElasticsearch folder under MyAppCreate files and subfolder according to the diagram
41index.html
angular-sanitize.js is used to handle html content showing on UIui-bootstrap-tpls-.js is used to show & control paginationelasticsearch.angular.js is used to connect Elasticsearch & submit query command
42
factories.jsdefine our elasticsearch cluster host IPs
use esFactory to connect elasticsearch clusters43
app.jsdefine our routing, UI template & controller
44fulltext-search.html (1)
45fulltext-search.html (2)
46fulltext-search.html (2)
47controllers.js
48Fulltext-Search DemoSelect 11_AngularWithElasticsearch/index.html and Hit F5 to runDemo Page
49Highchart , AngularJS ,Web API2 , SignalR2, Redis + Elasticsearch IntegrationIntegration with Entity FrameworkCopy 10_IntegrationWithRedis to 12_IntegrationWithElasticsearch 51Create New ElasticDashboardRepo.cs
Control Unique ID generation and management via Redis52
Switch RedisDashboardRepo to ElasticsearchDashboardRepoCopy RedisDashboardController.cs to ElasticsearchDashboardController.csSwitch our Repository from Redis to Elasticsearch53
Modify Our Angular ChartDataFactorySwitch angular $http communication end point to our new WebAPI urlBeforeAfter
54Integration with ElasticsearchSelect 12_Int