AWS Search ServicesEiji Shinohara

(2014A9CloudSearch)

{ "Name" : "", "Twitter" : "@shinodogg", "Profile" : { "Role" : "Solutions Architect", "Market": "Startup", "Services" : [ "Amazon CloudSearch", "Amazon Elasticsearch Service", "Amazon Simple Workflow Service, "AWS Elastic Beanstalk ] }}

New

AWS Amazon CloudSearch

https://aws.amazon.com/jp/cloudsearch/

Amazon Elasticsearch Service https://aws.amazon.com/jp/elasticsearch-service/

Amazon CloudSearch Amazon Elasticsearch Service

A9.com CloudSearch/Amazon ES

AmazonA9A9.com

Amazon CloudSearch

2011 API

A9 Amazon.com

2013 API on top of Apache Solr

n-gram 20143

Amazon CloudSearch

Auto Scaling / Auto Partitining

Auto Partitioning

Auto Scaling

Amazon CloudSearch

Amazon EMR

Index

Index P1

Index P2Amazon EMR

Amazon CloudSearch

schoo nanapi ChatWork SnapDish SmartInsight Lancers SmartNews

A9 x SmartNews in Palo Alto

CloudSearch Meetup at Amazon Meguro Office

SourceFieldDynamicField

SourceField Bi-Gram2

SolrCopy Field

SourceFieldDynamicField

DynamicField 1

SourceFieldDynamicField

DynamicField bar_txt

*_txt

(Morphological Analysis)

Amazon CloudSearch - Japanese Text Processing

(Morphological Analysis) : AWS CLI/SDK{ "AnalysisSchemeName": "jascheme", "AnalysisSchemeLanguage": "ja", "AnalysisOptions": { "JapaneseTokenizationDictionary": "[ [\\,\ \,\ \",\",,\]]" }}$ aws cloudsearch define-analysis-scheme --region us-east-1 --domain-name mydomain \--analysis-scheme file://jascheme.txt

Amazon CloudSearch - Japanese Text Processing

(Synonym Addition) Venice

Alias Pupil => Student Student => Pupil

Group 1st, first, one =>

CLI/SDK

http://ja.wikipedia.org/wiki/

Amazon CloudSearch - Japanese Text Processing

A/B

Amazon CloudSearch - Japanese Text Processing

Amazon CloudSearch

CloudSearch

CloudSearch5MB

Amazon CloudSearchAPI SolrSolrAPI Solr

Elasticsearch

Google Trends

ELK stack

ELK is Elasticsearch, Logstash, and Kibana

Elasticsearch is a distributed, schema-free search and analytics engine

Logstash is a tool for collecting and managing events and logs.

Kibana is a browser-based analytics and search dashboard for Elasticsearch.

Kibana

http://www.binpress.com/blog/2014/10/21/binpress-podcast-episode-16-shay-banon-elasticsearch/

Amazon Elasticsearch Service

Shay()

Amazon Elasticsearch Service

Elasticsearch AWS Elasticsearch

Elasticsearch/

Kibana Management ConsoleURL

EC2 EBSEBS

Amazon ES

Amazon Elasticsearch Service

http://aws.typepad.com/aws_japan/2015/10/amazon-elasticsearch-service.html

Amazon Elasticsearch Service

Elasticsearch

Apache Lucene(Apache SolrCloudSearch)

Kibana Amazon Elasticsearch Service

AWS1) CloudWatch LogsLambdaElasticsearchKibana2) DynamoDB StreamsLogstashElasticsearch AZ

Zone Awareness

Amazon Elasticsearch Service

Amazon ES CloudSearch

ElasticIAMCloudWatch

Amazon Elasticsearch Service

Amazon ESCloudWatch Metricsq Cluster status(Green/Yellow/Red)q Node countq Searchable documentsq Deleted documentsq CPU utilizationq Free storage spaceq JVM memory pressureq Automatic snapshot failuresq Master CPU utilization

q Master free storage spaceq Master JVM memory pressureq Read IOPSq Write IOPSq Read latencyq Write latencyq Read throughputq Write throughputq Disk queue depth

Amazon Elasticsearch Service

Amazon Elasticsearch Service

IAM Integration IP

) KibanaIP Signed requests with SigV4

) AWS Logstash: https://github.com/awslabs/logstash-output-amazon_es

Fine-grained ) Index

Logstash Amazon ESData Source

Amazon Elasticsearch Service

ICUKuromoji

analysis-icu() analysis-kuromoji()

(201510)

Amazon Elasticsearch Service

Deployment

node node

EBS EBS

node

node

nodeMaster Nodes Data Nodes

ClusterNode

Shard 3

node

EBS

node

EBS

Shard 1

Shard 4

Shard 2

Amazon Elasticsearch Service

AWS

Daily: 11 Automated snapshot start hour AWS

w/ _snapshot API ElasticsearchS3

Amazon Elasticsearch Service

2016

) Elasticsearch2.0Kibana4.2 9300Elasticsearch VPC

AmazonES