1. C5.2Cloudera

2. 20114Cloudera Cloudera email: sho@cloudera.com twitter: @shiumachi 2014 Cloudera, Inc. All rights reserved. 2 3. 2014 Cloudera, Inc. All rights reserved. 3Cloudera Enterprise CDH: Hadoop Cloudera Manager (CM) : Cloudera Director: CM+CDH Cloudera Navigator: 5CMCDHC5.0 CM5CDH5.x / 4.x 4. 2014 Cloudera, Inc. All rights reserved. 4CDH Clouderas Distribution including ApacheHadoop 100% 5. 2014 Cloudera, Inc. All rights reserved. 5Cloudera Manager CDH (Cloudera Express)(Cloudera Manager Enterprise Edition) 6. 2014 Cloudera, Inc. All rights reserved. 6Cloudera Director CM+CDH AWS 7. 2014 Cloudera, Inc. All rights reserved. 7Cloudera Navigator () 8. 2014 Cloudera and/or its affiliates. All rights reserved. 8CDH 9. 2014 Cloudera, Inc. All rights reserved. 9CDH 5.2 (1)9CDH 5.0.0CDH 5.2.0Apache Avro1.7.5+cdh5.0.0+161.7.6+cdh5.2.0+67Apache Crunch0.9.0+cdh5.0.0+210.11.0+cdh5.2.0+9DataFu1.1.0+cdh5.0.0+7 1.1.0+cdh5.2.0+13Apache Flume1.4.0+cdh5.0.0+111 1.5.0+cdh5.2.0+49Apache Hadoop 2.02.3.0+cdh5.0.0+548 2.5.0+cdh5.2.0+551Apache HBase0.96.1.1+cdh5.0.0+60 0.98.6+cdh5.2.0+55HBase Solr1.3+cdh5.0.0+38 1.5+cdh5.2.0+19Apache Hive0.12.0+cdh5.0.0+308 0.13.1+cdh5.2.0+221Hue3.5.0+cdh5.0.0+365 3.6.0+cdh5.2.0+509Cloudera Impala1.3.0+cdh5.0.0+0 2.0+cdh5.2.0+0Kite SDK0.10.0+cdh5.0.0+79 0.15.0+cdh5.2.0+150 10. 2014 Cloudera, Inc. All rights reserved. 10CDH 5.2 (2)10CDH 5.0.0CDH 5.2Llama1.0.0+cdh5.0.0+0 1.0.0+cdh5.2.0+0Apache Mahout0.8+cdh5.0.0+27 0.9+cdh5.2.0+17Apache Oozie4.0.0+cdh5.0.0+174 4.0.0+cdh5.2.0+321Parquet1.2.5+cdh5.0.0+91 1.5.0+cdh5.2.0+35Parquet-format1.0.0+cdh5.0.0+3 2.1.0+cdh5.2.0+4Apache Pig0.12.0+cdh5.0.0+27 0.12.0+cdh5.2.0+40Cloudera Search1.0.0+cdh5.0.0+0 1.0.0+cdh5.2.0+0Apache Sentry1.2.0+cdh5.0.0+71 1.4.0+cdh5.2.0+84Apache Solr4.4.0+cdh5.0.0+178 4.4.0+cdh5.2.0+282Apache Spark0.9.0+cdh5.0.0+31 1.1.0+cdh5.2.0+56 11. 2014 Cloudera, Inc. All rights reserved. 11CDH 5.2 (3)11CDH 5.0.0CDH 5.2.0Apache Sqoop1.4.4+cdh5.0.0+43 1.4.5+cdh5.2.0+47Apache Sqoop 21.99.3+cdh5.0.0+26 1.99.3+cdh5.2.0+30Apache Whirr0.9.0+cdh5.0.0+4 0.9.0+cdh5.2.0+10Apache ZooKeper3.4.5+cdh5.0.0+28 3.4.5+cdh5.2.0+82 12. 2014 Cloudera, Inc. All rights reserved. 12 RHEL5 5.1.0 Ubuntu Trusty 14.04 5.2.0 13. 2014 Cloudera, Inc. All rights reserved. 13HDFS HDFS (HDFS-5776)5.0.05.0.0 NFSv3 5.0.0 POSIX ACL5.1.0 HDFS (not production ready) (xattr) Web UI 5.1.05.2.05.2.05.2.0 14. 2014 Cloudera, Inc. All rights reserved. 14YARN Fair 5.2.0 Instantaneous Fair Share Fair Share 1001 fair share REST API kill5.2.0 15. 2014 Cloudera, Inc. All rights reserved. 15HBase 0.98 (5.1)5.1.0 (HBASE-7404) (5.1)5.1.0 Slabdeprecated(5.2) (5.1)5.1.0 MR (5.1)5.1.0 REST API (5.1)5.1.0 ACL (5.2)5.2.0 hbase.hstore.flusher.count 2 (5.2)5.2.0 hbase.hregion.memstore.block.multiplier4 (55.2.2).0 16. 2014 Cloudera, Inc. All rights reserved. 16Impala Impala Meetup http://www.slideshare.net/Cloudera_jp/impala-20-update-impalajp Impala 1.4 (5.1)5.1.0 ORDER BY without LIMIT impala-shell UTF-8 YARN (Llama) Impala 2.0 (5.2)5.2.0 (LEAD, LAG) GRANT / REVOKE (Sentry) text + gzip / bzip2 / snappy 17. 2014 Cloudera, Inc. All rights reserved. 17Cloudera Search Enron Email (5.1)5.1.0 solrctl Solr (5.1.05.1) Parquet (5.1)5.1.0 CrunchIndexerTool Spark (5.2)5.2.0 Hive SolrStorageHandler (5.2)5.2.0 (5.2)5.2.0 (5.2)5.2.0 Deep Paging (5.2)5.2.0 18. 2014 Cloudera, Inc. All rights reserved. 18CDH Apache Spark: 1.1 (5.2)5.2.0 Apache Flume: Kafka (5.2)5.2.0 Hue: (LT) 19. 2014 Cloudera and/or its affiliates. All rights reserved. 19Cloudera Manager 20. 2014 Cloudera, Inc. All rights reserved. 20(CM4) 21. 2014 Cloudera, Inc. All rights reserved. 21(CM5) 22. : Cloudera Manager 2014 Cloudera, Inc. All rights reserved. 22 CM4.5 root rpmtar.gz CM5 23. 2014 Cloudera, Inc. All rights reserved. 23 HDFS: IO, , etc MapReduce: Map, Reduce 24. 2014 Cloudera, Inc. All rights reserved. 24 25. 2014 Cloudera, Inc. All rights reserved. 25 26. 2014 Cloudera, Inc. All rights reserved. 26CM CM5 27. 2014 Cloudera, Inc. All rights reserved. 27 IT: 20%: 30% UI YARN ImpalaYARN GUI 28. 2014 Cloudera, Inc. All rights reserved. 28 : CPUI/O cgroup (Linux ) (Java ) : HBaseHDFSImpalaMapReduceSolrSparkYARN 29. 2014 Cloudera, Inc. All rights reserved. 29 YARNImpala : :9-17 30. 2014 Cloudera, Inc. All rights reserved. 30 31. CM Sentry () (5.1) Isilon (5.2) KMS (key management server) (5.2) Key Trustee (Navigator) (5.2) Spark (on YARN) (5.2) 5.1 Spark Spark (Standalone) 2014 Cloudera, Inc. All rights reserved. 315.1.05.2.05.2.05.2.05.2.0 32. 2014 Cloudera and/or its affiliates. All rights reserved. 32 33. 2014 Cloudera, Inc. All rights reserved. 33EOM (End of Maintenance) EOM = EOM CM4 / CDH4 CDH EOM DateCDH3 2013/6/20 ()CDH4 2015/6/4CDH5 2017/4/2Cloudera Manager EOM DateCM 4.0 / 4.1 2014/9/1 ()CM 4.5 / 4.6 / 4.7 / 4.8 2015/6/4CM 5 TBD 34. C5.2