Jubakit ®ç´¹»‹

  • View
    5.674

  • Download
    2

Embed Size (px)

Text of Jubakit ®ç´¹»‹

  • Jubakit Kenichi Maehashi

    Jubatus Casual Talks #4 June 18, 2016

  • INTRODUCTION TO JUBAKIT

    Jubakit

    Jubatus Python scikit-learn

    Python 2.6 , 3.2 Jubatus 0.8.0

  • INTRODUCTION TO JUBAKIT

  • INTRODUCTION TO JUBAKIT

    Jubakit

    scikit-learn Jubatus

    Jubatus ()

    Jubatus (RPC , , , etc.) ()

  • INTRODUCTION TO JUBAKIT

    Jubatus :

    1. Jubatus

    2. Jubatus

    3.

    4. Jubatus

    Jubatus (JSON)

    5.

    /

    RPC

  • INTRODUCTION TO JUBAKIT

    Jubatus : Jubakit

    1. Jubakit 2. 3.

    4.

    1

    Jubatus

    scikit-learn

    RPC

  • INTRODUCTION TO JUBAKIT

    (CSV )

    dataset=Dataset(CSVLoader(dataset.csv))service=Anomaly.run(Config())forresultinservice.add(dataset):print(result)

    Jubakit

    : : :

  • INTRODUCTION TO JUBAKIT

    (CSV )

    dataset=Dataset(CSVLoader(dataset.csv))service=Anomaly.run(Config())forresultinservice.add(dataset):print(result)

    CSV

  • INTRODUCTION TO JUBAKIT

    (CSV )

    dataset=Dataset(CSVLoader(dataset.csv))service=Anomaly.run(Config())forresultinservice.add(dataset):print(result)

  • INTRODUCTION TO JUBAKIT

    (CSV )

    dataset=Dataset(CSVLoader(dataset.csv))service=Anomaly.run(Config())forresultinservice.add(dataset):print(result)

  • INTRODUCTION TO JUBAKIT

    Jubakit

    LOADER

    DATASET

    SCHEMA

    SERVICE

    CONFIG

    CSV

    RDBMS

    TWITTER

    (, , ETC.)

    NUMPY

    SCIPY

    (Estimator)

  • INTRODUCTION TO JUBAKIT

    Loader - (1/2)

    Loader

    LineBasedStreamLoader / LineBasedFileLoader

    /

    ArrayLoader / ZipArrayLoader

    ( NumPy Array)

    SparseMatrixLoader SciPy 2-D

    CSVLoader CSV

    TwitterStreamLoader Twitter

    MergeChainLoader Loader

    ValueMapChainLoader Loader

  • INTRODUCTION TO JUBAKIT

    Loader - (2/2)

    BaseLoader Loader

    . 1 yield rows()

    Loader () Loader preprocess()

  • INTRODUCTION TO JUBAKIT

    Schema -

    Loader /

    STRING

    NUMBER

    BINARY

    LABEL (Classifier )

    AUTOLoader (CSV Loader )

    INFER

    IGNORE

  • INTRODUCTION TO JUBAKIT

    Schema -

    schema = Schema({

    Category: Schema.LABEL,

    Title: Schema.STRING,

    Year: Schema.NUMBER

    }, Schema.IGNORE)

    Category Title Year Country DirectorSF Ep1 1999 USA G. Lucas

    2001 Japan H. Miyazaki 1997 USA J. Cameron

    SF 1991 USA J. Cameron

    :

  • INTRODUCTION TO JUBAKIT

    Service -

    Dataset scikit-learn Estimator ( BaseEstimator ) Jubatus

    Classifier (), Anomaly (), Weight () 3

  • INTRODUCTION TO JUBAKIT

    $ pip install jubakit

    jubakit scikit-learn

    $ pip install numpy spicy scikit-learn

    Jubakit

    Jubatus

    pip (yum/apt-get)

  • INTRODUCTION TO JUBAKIT

    :

    Kaggle Titanic

    https://www.kaggle.com/c/titanic

    https://github.com/kmaehashi/jubakit-tutorial-titanic

    https://www.kaggle.com/c/titanichttps://github.com/kmaehashi/jubakit-tutorial-titanic

  • INTRODUCTION TO JUBAKIT

    example scikit-learn

    Twitter

  • INTRODUCTION TO JUBAKIT

    Jubakit Loader RDBMS MQ AWS SQS Kinesis

    Service (Recommender, NN, ) etc.