Documentbd

A  Big  Data  Business  

James  Yu  (虞沐)    @Huami  US  (华⽶米美国)  

Who  am  I?  

•  14  years  in  so?ware  industry  – 10  years  backend  engineer  – 4  years  Big  Data  and  Cloud  CompuIng  

•  6  years  in  China  (HP/SAP),  and  8  years  in  Silicon  Valley  (eBay/Samsung/Baidu/LinkedIn)  

•  Now  @Huami  US,  as  architect  and  manager  of  cloud  and  big  data  team  

Agenda  

•  What  is  a  Big  Data  business  •  How  to    •  China  VS.  US  •  A  story  of  healthcare/wearable  •  How  we  do  it  

What  is  a  Big  Data  business  

•  Free  services  (or  even  give  away  money)  •  Lots  of  users  •  Knowing  what  each  individual  wants  •  Making  money  by  connecIng  people  and  business  

(Google,  Facebook,  TwiZer,  Yelp,  LinkedIn,  Uber,  Airbnb,  BAT,  嘀嘀出⾏行,  美团,  饿了么)  

How-­‐to  

•  Business  model    (great  services  that  people  like)  

•  Talents  and  technologies  (right  people,  right  technology)  

•  Good  luck  (more  than  50%  big  data  projects  fail)  

China  VS.  US  

China   Items   US  

market  

compeIIon  

talents  

tech  -­‐  infra  

tech  -­‐  ecosystem  

$  investment  

Wearable  and  Healthcare  

Name:  Tom    Walking  steps/day:  2000  Running  duraIon:  30  minutes  Sleeping  score:  75  Heartbeat  min/median/max:  70/80/90    RecommendaIons:  Follow-­‐up  for  heartbeat  More  walking  needed  

Wearable  and  AuthenIcaIon  

Wearable  connects  everything  

Huami  

•  MiBand-­‐1  @2014/8  @79  RMB  •  10M  users,  2rd  worldwide,  next  to  Fitbit  •  Amazfit  @2015/9  @299  RMB  

•  More  coming  in  next  few  months  

Huami  –  how  we  do  it  

•  Business  model:  low  cost  band  •  Technology:  wearable  hardware,  user  app,  and  robust  cloud  with  big  data  system  

•  Talent:  3  locaIons  

Cloud  CompuIng  

Our  Technology    -­‐    Cloud  

OpIons:  •  AWS  •  Google  Cloud  •  Microso?  Azure  •  Aliyun  

Compare:  Pros  &  Cons  AWS   Google   MS  Azure   Aliyun  

Global  coverage     Yes   Yes  

China  coverage   Yes  

Features   Yes   Yes  

Open  source  friendly  

Yes   Yes  

Quality  and  stability  

Yes  (since  2006)  

Yes   Yes  

New  features   Yes  

Price   Yes   Yes   Yes  

DocumentaIon   Great   OK   OK   Poor  

Our  choice  is  AWS  

•  AWS  has  the  best  technology  and  quality.  •  AWS  has  the  best  global  coverage.  •  Aliyun  has  the  best  coverage  in  China.  

Global  datacenters  

US   China  

EU  

Singapore  

AWS  offerings  -­‐  regions  

China: China

Asia: ap-southeast-1 (ap-southeast-2, ap-northeast-1)

US: us-east-1 (us-west-2)

EU: eu-central-1 (eu-west-1)

AWS  offerings  -­‐  services

Big  Data  

Big  Data  steps  

Big  Data  Lambda  Architecture  

Big  data  -­‐  opIons  Component   Op;ons  

Real  Ime  data  processing   •  Spark  streaming  (recommended)  •  Storm  

Real  Ime  NoSQL  database   •  DynamoDB  (recommended)  •  Cassandra  •  HBase  •  MongoDB  

Offline  data  storage   •  S3  (recommended)  •  HDFS  

ETL  (SQL)   •  Hadoop  Hive/Pig    •  Spark  SQL  (recommended)  

ETL  (programming)     •  Hadoop  MapReduce  •  Spark  RDD/Dataframe  programing  

(recommended)    

Batch  AnalyIcs   •  Spark  (SQL)  •  Redshi?  (recommended)      •  Hadoop  Hive/Pig  

Machine  learning   •  Hadoop  Mahout  •  Spark  MLlib  /  SparkR  (recommended)    •  AWS  machine  learning  •  R  /  SciPy  /  Matlab  /  DeepLearning  

Data  products  

•  Helping  user  to  beZer  track  his/her  acIvity,  includes  fitness  and  health  

•  Develop  an  ecosystem  with  partners  from  different  areas  (smart  appliance,  security,  payment,  and  many  others)  

Q  &  A