Click here to load reader

Elasticsearch x Autoscaling(AWS)

  • View
    1.275

  • Download
    4

Embed Size (px)

Text of Elasticsearch x Autoscaling(AWS)

  1. 1. Amazon Web Searvice elasticsearch
  2. 2. DMPAudienceSearch
  3. 3. IMelasticsearch elasticsearch AutoScaling
  4. 4. elasticsearch ID User Agent IP ID
  5. 5. AudienceSearch 1000 30
  6. 6. AWS Cluster: 1, Type: 1, Node: 86 master: 3 (master:true, data: false) data: 80 (master:false, data:true) searcher: 2 (master:false, data: false) indexer: 1 (master:false, data: false) version: 1.3.8 1.5.2 (5/26)
  7. 7. masterdatasearcher x2 indexer indexerAutoScaling x3x80 ELB
  8. 8.
  9. 9. Cluster Size Shard = Node 10 200 100 200Aggregation 80
  10. 10. Shard Node = 80 c3.large (1) shard: 40, replica: 1 shard/node = 0.5 (2) shard: 80, reprica: 1 shard/node = 1 (3) shard: 160, replica:1 shard/node = 2 shard/node shard/nodeshard/cpu
  11. 11. Instance Size SSD Ephemeral Diskc3 xlargeCP largexlarge30GB80GB JVMdoc_value
  12. 12. AutoScalling
  13. 13. Why AutoScaling ? AutoScaling terminate
  14. 14. recovery LifeCycle recovery 0 indices: recovery: max_bytes_per_sec: 500mb concurrent_streams: 4
  15. 15. recovery LifeCycle : 40GB 10c3.xlarge
  16. 16. elasticsearch-cloud-aws discoverymaster node SG discovery: type: ec2 ec2: tag: es_cluster: production
  17. 17. master node aws:autoscaling:groupName
  18. 18. ELB ELB master, data 92009300
  19. 19.
  20. 20. Multi-AZ3 shardreplicaAZ cluster: routing: allocation: awareness: attributes: aws_availability_zone
  21. 21. aws_availability_zoneAZ elasticsearch-cloud-aws { "attributes" : { "aws_availability_zone" : "ap-northeast-1c", "master" : "false" } }
  22. 22. hostname hostname(ip-10-0-0-1)ES AMI node: name: ${HOSTNAME} export HOSTNAME=$(hostname -s) /etc/syscong/elasticsearch /etc/elasticsearch/elasticsearch.yml
  23. 23. ELB Mackerel Papertrail /Slackwebhook
  24. 24. Papertrail master data searcher x2 x3 x80 SSHtail syslog
  25. 25. AWSES AutoScalling Cluster Size 1indexelasticsearch
  26. 26.
  27. 27.

Search related