2015 Teradata 大数据峰会·深圳

3

Nous Introduction

Tech Stack behind Nous

Agenda

© 2014 Teradata 3

4

Why build NOUS

Where to find data? Where to find complete data? Which report/number is trust

worthy?

Email/Tableau

Ask help

Can’t get data in time! Can hardly run query against behavior data! Can only get basic data without advanced

support!

PM Analyst

One stop shop! Single source of truth!

Providing advanced analytics functions! Prompt response based on inverted index on top of Hbase & TD!

Unified Experience

4

5

Quick

Strike

Nous-PHD

Nous-Behavior Analytics

Nous Domain, EP Report,

Metrics Explorer

Tableau

Hive, SQL, Metrics Explorer

TD UDF, Hadoop Lib, Data Service

TD, Hadoop, Hbase, Kylin

What is Nous ?

Understand business operation

Performance

Monitor key

product metrics

Learn user behavior

cross domains

Explore/size

opportunity

A/B test analysis

Develop prototypes

Serve one time

reports

Explore data

interactively

Facilitate data

processing

Enable data

feature Keep

data

“After years of struggling with half

finished or unsupported behavioral

analytics tools, Nous is something that

truly gets me excited. --Ken Chin, Manager, Product Management 3, EPD

UK PD HC

I can’t believe how good this is! It’s

soooooo good! - Eric Anderson Site Email Marketing Senior Manager,

Loyalty Mktg

NOUS fills a gap between free-form queries of “big data”

and business metrics

5

6

6.41%

8.89%

15.31

%

17.27%

29.92

%

33.22

%

Others

Engineer

Executive &

Management

BU

Analyst &

Scientist

Product

Manager

0.00%5.00%10.00%15.00%20.00%25.00%30.00%35.00%

2014 Accumulated Unique Users by

User Country Penetration Rate by User Persona

Who is using NOUS

2,256 Users in 2014 | 358 Users returns every month | 2.5 X User

Growth from May to Dec

6 Launched in May 2014

What can Nous do ?

7

8

8

Nous Landscape

Behavior

Analytics

Domain

Deep Dive

Product

Health

Shopping Experience

Local & Selling

Ads & Monetizatio

n

Payment, Trans & Finance

Customer Connection

iPad VQ: Does channel order really matter?

9

10

Traffic by VQ Channels

#4 channel electronics gets half the visits as the #2 channel fashion,

Order Matters! Should we provide personalized channel

order?

10

Which part of iPad VQ is most popular?

11

12

VQ Heatmap 12

People like to browse on Motors and check Sales on Collectible Art. Should we move

browse upward on Motors?

What type of content in VQ is more engaging?

13

14

Sales Collection

Different Contents Drive Different Engagement

14

Users take different paths, 37% people viewed items after Collection vs

22% after Sales, 11% people did search after Sales vs 8% after

Collection! Need to improve Sales quality!

Does people like the content in the channel or they want see more?

15

16

Collections Sales Deals

Clicks on Displayed Content vs “More” link

CTR(%) Collections Sales Deals

Default Displayed Content 1.28% 4.27% 0.56%

The "More" Link 0.44% 0.25% 0.66%

Ratio (More/ Default Content) 0.34 0.06 1.18

16

People want to see more deals! Can we provide

personalized deal as default content?

Is there poor performing content in VQ?

17

18

Poor Content Leads to Poor Engagement

18

Featured collection’s conversion dropped significantly

during Jan 3-4, which is caused by that most items in that

collection ended!

What are the key metrics changed by VQ?

19

20

Product Health – Cart Share of Transactions

20

Changing color of “Add to Cart” button significantly increased the cart share of

transactions!

What is the biggest opportunity for Merch?

21

22

Merch modules GMB distribution (as of June 2014) 22

Seller Other Item Module is the highest GMB Merch Module which hadn’t been optimized before people see this chart!

Tech Stack behind NOUS

23

24

24

Component Details

25

26

eBay CDL(Common Data Layer) framework

A distributed data processing framework based on MAP/REDUCE.

Mainly used to analyze Sojourner ( user access log in eBay both from Web and Mobile App) data.

Ability to handle massive data on cluster.

Abstract business logic into separate units, which is standardized, reusable.

Easily join data on HDFS and Teradata by working with TDBridge together.

TDBridge

The bridge is a framework for moving data in and out of Teradata

Through SQL a user is able to import, export, stream data via the bridgeSupport multiple data formats on HDFS, (CSV, Avro, Sequence file …)

A common use of the bridge is to move data between Teradata and Hadoop. The bridge is bi-directional. It is always instantiated via SQL.

Traditional Batch(Java, Shell Scripts)

26

Data Processing Layer

27

H2OLAP Solution A lightweight OLAP Solution based on Hbase.

Used reverted index technology.

Handle multiple dimensions data easily.

Restful API to access both meta info and data set.

Support more than 20 dimensions, billion of rows data set in real production environment. High cardinality dimension support (sellerid dimension), 80% queries returned in 5 seconds.

Apache Kylin http://www.kylin.io/

Designed to reduce query latency on Hadoop for 10+ billions of rows of data

Extremely Fast OLAP Engine at Scale

Offers ANSI SQL on Hadoop and supports most ANSI SQL query functions

Offers integration capability with BI Tools like Tableau

Microstrategy CUBE

Tableau Server

27

Query Engine

28

Using R software

Support multiple Models/Algorithms.

Support user subscription.

Input: fetching data from various data sources directly, including H2OLAP/KylinIO and Teradata

Output: Alerts/Warnings are saved into relational database. Downstream applications

fetch/dispatch these alerts/warnings from DB.

28

Alert Engine

29

eBay Nous VizFabric Engine

Secure – integrated with CORP LDAP

Unified Backend service – talk with backend service through restful API/predefined interface

Unified look and feel

Highly reusable UI Component and Page Template (Reduce development effort, deliver new product very fast)

Microstrategy Web

Running query on Microstrategy CUBE

Tableau Desktop

Fetching data from Tableau Server.

Sharing report/dashboard within organization.

29

Visualization Engine

Q&A

30

31

Appendix -- The Journey of Nous

31

32

Appendix -- Example of content performance

optimization

Super Bowl w/

Katty Perry

New York Fashion

Week

Fe

b

Identify Topic Social calendar

Source

Inventory Ranking & Validation

Optimize Performance Real Time monitor

Action recommendations

Software:Personal

Finance, Tax &

Legal

Collectibles:Holiday

& Seasonal:St.

Patrick's Day

32

33 33 © 2014 Teradata