S Q LORNoSQL

That’sT H E Ques.on

@majidfnMajiD FatemianUbisoft Montreal

BIGDATA

NoSQL

1 2 3

No SQL Cluster Schemaless

Fitness Evolved

Fitness Evolved

2.2M 10K 160K(Max)40K (Avg)

Users Concurrent Daily Activities

STATS

2011

2012

2013

2011

2012

2013

Architecture2011

SQL / JOINACID

TransactionPartitioningReplication

ToolsOPs

Data Modeling

+

2011

2012

2013

Profile Info.

SpecificGlobal

Aggregated stats Activities

DATA

Community Events

Details

Leaderboards

SQL / JOINACID

TransactionPartitioningReplication

ToolsOPs

Data Modeling

VolumeNo DetailsVariant SchemaPartitioningScale outSharding

+ -

SQL / JOINACID

TransactionPartitioningReplication

ToolsOPs

Data Modeling

VolumeNo DetailsVariant SchemaPartitioningScale outSharding

+ -{ player_id: 1234, game_id: 4001, platform: xbox360, calories: 40, duration: 75, difficulty_level: medium, jumping_ropes: 178, score: 450, endurance:3, category: 4}

{ player_id: 1234, game_id: 1002, platform: wiiu, calories: 75, duration: 120} *

***

xbox v.1 xbox v.2Variant Schema

SQL / JOINACID

TransactionPartitioningReplication

ToolsOPs

Data Modeling

VolumeNo DetailsVariant SchemaPartitioningScale outSharding

+ -Scale out

SQL / JOINACID

TransactionPartitioningReplication

ToolsOPs

Data Modeling

VolumeNo DetailsVariant SchemaPartitioningScale outSharding

+ -Sharding

Data CData B

Data A

Data Access Layer

SQL / JOINACID

TransactionPartitioningReplication

ToolsOPs

Data Modeling

VolumeNo DetailsVariant SchemaPartitioningScale outSharding

+ -

NoSQL

...

mongoDB

1 2 3

OPs Support Community

Schema-LessAggregated DataObject MappingLarge Scale Data

ShardingMap/Reduce

Eventual ConsistencyMemory Storage

Journaling

+

Aggregated DataObject MappingLarge Scale Data

ShardingMap/Reduce

Eventual ConsistencyMemory Storage

Journaling

Schema-Less

{ player_id: 1234, game_id: 4001, platform: xbox360, calories: 40, duration: 75, difficulty_level: medium, jumping_ropes: 178, score: 450, endurance:3, category: 4}

{ player_id: 1234, game_id: 1002, platform: wiiu, calories: 75, duration: 120} *

***

xbox v.1 WiiU

+

Schema-Less

Large Scale DataSharding

Map/ReduceEventual Consistency

Memory StorageJournaling

Aggregated DataObject Mapping

{ player: { name: example, gender: M, country:CA } game_id: 4001, date: “2013-11-08”, calories: 40, duration: 75,}

+

Schema-LessAggregated DataObject Mapping

Eventual ConsistencyMemory Storage

Journaling

+

Large Scale DataSharding

Map/Reduce

Data AData B

Data C

Schema-LessAggregated DataObject MappingLarge Scale Data

ShardingMap/Reduce

Eventual ConsistencyMemory Storage

Journaling

+

Denormalized dataDisk SpaceExpertiseComplex QueryingResource UsageDB-Level Locking

-

Schema-LessAggregated DataObject MappingLarge Scale Data

ShardingMap/Reduce

Eventual ConsistencyMemory Storage

Journaling

+

BENCHMARK

Random ProfileSearch

(1M)

Full HistoryRetrieval

(6M)

Insert(6M)

AggregationMap/Reduce

(6M)

0

0.005

0.01

0.015

0.02

MySQLMongoDB

Relational Data ModelEase of Use

Schema LessPerformanceSharding

GO!Hybrid

Master

Slave

Config Server

Shard 1Replicaset

Shard 2Replicaset

Concerns

Inconsistency!

2

3

4

Data Duplication

Consistency vs. Latency

Complexity

Challenges

Availability

! 2

Time

3

Expertese

Data Access Layer

OR

FailureBackup

Plans

DATAMigration

1 One Shot Migration

2 Profile Versioning

4 hrs !!Live!

Read

Write

TEST!

2Functional

3Integration

1Unit

FinalResults

Scalability

&...

B.I.User

Details

Response Time

0.02 0.0002” ”

Architecture2013

REST

> 1 year uptime

1

2

Based on Requirements

Benchmark!

3 Choose Wisely!

ThankYOU!

@majidfnhttps://joind.in/9977

References

NoSQLDistilled

MongoDBThe definitive guide

MySQLHigh Performance

we needYOU!

http://montreal.ubisoft.com/en/video-games-jobs-my-career

@majidfn

Q?Got