機械学習は、増え続けるデータをもとに、事業戦略の判断やより正確な予測、関連性の推定を行うための、重要なツールです。機械学習の中でも、最も幅広く活用されているアプリケーションはレコメンデーションエンジンです。スケーラブルな機械学習ライブラリであるMahoutは、レコメンデーションの生成とデータの扱いをシンプルなものにしてくれます。本講演では、より構築が簡単なレコメンデーションエンジンのデザインと、そのイノベーティブな実装方法を活用した場合の利点を紹介します。2014年7月8日に開催されたHadoop Conference Japan 2014での講演資料です。
- 1. 2014 MapR Technologies 1 2014 MapR Technologies MahoutSolr
MapR Technologies
2. 2014 MapR Technologies 2 ? ? ? 3. 2014 MapR Technologies 3
eBook Ted Dunning, Ellen Friedman
http://hadoop-times.com/ml-pdf-ebook.html Apache Mahout Solr 4.
2014 MapR Technologies 4 2014 MapR Technologies 5. 2014 MapR
Technologies 5 ? 6. 2014 MapR Technologies 6 A1 A2 ! " # $ T A1 A2
! " # $= A1 T A2 T ! " % % # $ & & A1 A2 ! " # $ = A1 T A1
A1 T A2 AT 2A1 AT 2A2 ! " % % # $ & & r1 r2 ! " % % # $
& & = A1 T A1 A1 T A2 AT 2A1 AT 2A2 ! " % % # $ & &
h1 h2 ! " % % # $ & & r1 = A1 T A1 A1 T A2 ! "% # $& h1
h2 ! " % % # $ & &O( k d + k3 d) = O(k2 d log n + k3 d) for
small k, high quality O( d log k) or O(d log log k) for larger k,
looser quality 7. 2014 MapR Technologies 7 : : e 8. 2014 MapR
Technologies 8 Google Maps: 9. 2014 MapR Technologies 9 Google
Maps: 10. 2014 MapR Technologies 10 2014 MapR Technologies (I want
a pony) 11. 2014 MapR Technologies 11 I want a pony! 12. 2014 MapR
Technologies 12 : 13. 2014 MapR Technologies 13 14. 2014 MapR
Technologies 14 15. 2014 MapR Technologies 15 16. 2014 MapR
Technologies 16 ? ? 17. 2014 MapR Technologies 17 ! 18. 2014 MapR
Technologies 18 19. 2014 MapR Technologies 19 20. 2014 MapR
Technologies 20 ? ? ? 21. 2014 MapR Technologies 21 ... ? 22. 2014
MapR Technologies 22 ( ) : () 23. 2014 MapR Technologies 23 x : x
(LLR) Apache Mahout RowSimilarityJob LLR 24. 2014 MapR Technologies
24 Apache Mahout Apache http://mahout.apache.org Mahout version 0.9
20142 Apache Hadoop Hadoop & Mahout Math 25. 2014 MapR
Technologies 25 26. 2014 MapR Technologies 26 u1 u3 u2 u1 u3 u2 u1
t1 t4 t3 t2 t3 t3 t1 27. 2014 MapR Technologies 27 : / 28. 2014
MapR Technologies 28 : / - 1 2 1 1 1 1 2 1 ?. 0 0 0 0 29. 2014 MapR
Technologies 29 : / LLR - 1 2 1 1 1 1 2 1 0 0 0 0 30. 2014 MapR
Technologies 30 1 1not not 1 31. 2014 MapR Technologies 31 A not A
B 13 1000 not B 1000 100,000 A not A B 1 0 not B 0 2 A not A B 1 0
not B 0 10,000 A not A B 10 0 not B 0 100,000 ? 32. 2014 MapR
Technologies 32 LLR Apache Mahout RowSimilarityJob LLR A not A B 1
0 not B 0 2 A not A B 1 0 not B 0 10,000 A not A B 10 0 not B 0
100,000 1.95 4.52 14.3 ? A not A B 13 1000 not B 1000 100,000 0.90
33. 2014 MapR Technologies 33 : LLR - 1 2 1 1 1 1 2 1 0 0 0 0 34.
2014 MapR Technologies 34 : : 35. 2014 MapR Technologies 35 ? 36.
2014 MapR Technologies 36 : 37. 2014 MapR Technologies 37 38. 2014
MapR Technologies 38 Apache Solr/Apache Lucene Apache Solr/Lucene:
Lucene Solr Lucene Web Web 39. 2014 MapR Technologies 39 LucidWorks
Solr/Lucene Solr MapR LucidWorks Solr Solr Lucene 40. 2014 MapR
Technologies 40 Solr/Lucene/LucidWorks 41. 2014 MapR Technologies
41 field:(1 2) field:(1 OR 2) field:(1 2 ~ 5) field:(1 AND 2) 42.
2014 MapR Technologies 42 id: t4 title: puppy desc: The sweetest
little puppy ever. keywords: puppy, dog, pet indicators: (t1) 1
Solr : Solr 43. 2014 MapR Technologies 43 : 44. 2014 MapR
Technologies 44 45. 2014 MapR Technologies 45 : Take the A Train
46. 2014 MapR Technologies 46 47. 2014 MapR Technologies 47 48.
2014 MapR Technologies 48 MusicBrainz IDMusicBrainz ID / 49. 2014
MapR Technologies 49 : 119 ? Duke Ellington ID ID ID ID 50. 2014
MapR Technologies 50 50 51. 2014 MapR Technologies 51 51 52. 2014
MapR Technologies 52 Lucene Mahout indicator id 541902 mbid
983d4f8f-473e-4091-8394-415c105c4656 ame Charlie Winston area
United Kingdom gender None Indicator_artists
997727,815,830794,59588,900,2591,1344,696268 id 1710 mbid
592a3b6d-c42b-4567-99c9-ecf63bd66499 ame Chck Berry area United
States gender Male Indicator_artists
386685,875994,637954,3418,1344,789739,1460 53. 2014 MapR
Technologies 53 Mahout 54. 2014 MapR Technologies 54 MapR Mahout
Web Pig Python NFS NFS NFS Python MapR 55. 2014 MapR Technologies
55 h () Ah AT Ah( ) AT A( )h () () A 56. 2014 MapR Technologies 56
AT Ah( ) AT A( )h 1 2 (h) 57. 2014 MapR Technologies 57 2014 MapR
Technologies 58. 2014 MapR Technologies 58 59. 2014 MapR
Technologies 59 60. 2014 MapR Technologies 60 (A) ( = , = ) (B) ( =
, = ) ATA BTB 61. 2014 MapR Technologies 61 BTA (?) () 62. 2014
MapR Technologies 62 : Paco de Lucia () : hombres de paco (TV) 400
: Van Halen 63. 2014 MapR Technologies 63 ? 64. 2014 MapR
Technologies 64 AT A( )h BT A( )h 65. 2014 MapR Technologies 65 A
66. 2014 MapR Technologies 66 A1 A2 ! " # $ 1 2 67. 2014 MapR
Technologies 67 A1 A2 ! " # $ T A1 A2 ! " # $= A1 T A2 T ! " % % #
$ & & A1 A2 ! " # $ = A1 T A1 A1 T A2 AT 2A1 AT 2A2 ! " % %
# $ & & r1 r2 ! " % % # $ & & = A1 T A1 A1 T A2 AT
2A1 AT 2A2 ! " % % # $ & & h1 h2 ! " % % # $ & & r1
= A1 T A1 A1 T A2 ! "% # $& h1 h2 ! " % % # $ & & 68.
2014 MapR Technologies 68 : 69. 2014 MapR Technologies 69 70. 2014
MapR Technologies 70 71. 2014 MapR Technologies 71 72. 2014 MapR
Technologies 72 ? 73. 2014 MapR Technologies 73 floor(t/T) s = logr
+ N(0,log) 1.5,3[ ] r r 74. 2014 MapR Technologies 74 = 2 1 2 8 3 9
15 7 6 1 8 14 15 3 2 22 10 1 3 8 2 10 5 7 4 1 2 10 7 3 8 6 14 1 5
33 15 2 9 11 29 1 2 7 3 5 4 19 6 1 3 5 23 9 7 4 2 2 4 11 8 3 1 44 9
2 3 1 4 6 7 8 33 3 4 1 2 10 11 15 14 11 1 2 4 5 7 3 14 1 8 7 3 22
11 2 33 75. 2014 MapR Technologies 75 : 76. 2014 MapR Technologies
76 Q&A @mapr_japan maprjapan [email protected] MapR maprtech
mapr-technologies
