Storage Methods for Nonstandard Data Patterns

Storage Methodsfor

Nonstandard Data PatternsB O B B U R G E S S

D ATA B A S E O P E R AT I O N S E N G I N E E R

2

AgendaIntro

Choices Available To You

Choices I Tested

Testing Methods

What I Saw When Loading Data

What I Saw When Using Data

Storage Engine Comparison

Conclusions

3

W H O I S T H I S G U Y ?

4

Bob Burgess

Developer, then DBA

1997, Oracle & Sybase

2007, MySQL at Radian6 => Salesforce

2015, MySQL at Shopify

[email protected]

2006 Platform to let merchants easily sell online and in person 243,000+ Active stores / $14B in sales / 150 Countries Large MySQL Installation Hiring!

5

6

Dont Panic

All slides will be posted to the conference site & Slideshare

Test scripts and test data will be posted to Github bobburgess

[email protected]

7

Why Am I Doing This?

I cant be the only one

Surprising results

8

C H O I C E S

9

Normal Tables

A few numbers and average-size varchar columns

Couple of indexes

Artificial or random primary key

10

Unusual Tables

Very tiny rows

Very large rows

Large random primary key

Unusual Tables Very tiny rows

Very large rows


11

Multiple TB

Two bigints

Generally-increasing primary key

InnoDB

Unusual Tables Very tiny rows

Very large rows


12

De-duplication

SHA1 hash

binary(20)

Completely random primary key

InnoDB

Unusual Tables Very tiny rows Very large rows


13

De-duplication

SHA1 hash

binary(20)

Completely random primary key

InnoDB

Yves Trudeau (Percona)

Auto-increment PK

Index on 1st 5 bytes of binary(20) hash

Trigger to check for Duplicates

14

Storage Engines

MySQL

client

disk

15

Storage Engines

MySQL

client

disk

MySQL

client

disk

Storage Engine

16

Storage Engines

MySQL

client

disk

MySQL

client

disk

Storage Engine

17

Storage Engines

MySQL

client

disk

MySQL

client

disk

Storage Engine

18

Choices Storage: Solid-state vs. Spinning Rust

Arrays of Storage (RAID levels)

Storage Engine

suitability for data shape

suitability for access patterns

adaptability to slower storage

19

MY T E S T SC H O I C E S T E S T E D ( VA R I A B L E S )

20

Primary Key

Auto-Increment (Sequential) Integer

Random Integer

Random Large (20 bytes)

Auto-Increment (Sequential) Integer With Trigger

21

Data Shape

22

Data Shape: Tiny Rows

create table tinyrows ( id bigint primary key, payload bigint);

23


create table tinyrows ( id bigint auto_increment primary key, payload bigint);

24


create table tinyrows ( id bigint auto_increment primary key, payload bigint, index i (payload));

25

Data Shape: Average Rowscreate table averagerows ( id bigint primary key, num0 bigint, char1 varchar(60), num1 decimal (10,2), num2 double, char2 varchar(60), char3 varchar(50));

26

Data Shape: Average Rowscreate table averagerows ( id bigint auto_increment primary key, num0 bigint, char1 varchar(60), num1 decimal (10,2), num2 double, char2 varchar(60), char3 varchar(50));

27

Data Shape: Average Rowscreate table averagerows ( id bigint auto_increment primary key, num0 bigint, char1 varchar(60), num1 decimal (10,2), num2 double, char2 varchar(60), char3 varchar(50), index i1 (num0), index i2 (char1), index i3 (char3), index i4 (num2));

28

Data Shape: Average Rowscreate table averagerows ( id binary(20) primary key, num0 bigint, char1 varchar(60), num1 decimal (10,2), num2 double, char2 varchar(60), char3 varchar(50), index i1 (num0), index i2 (char1), index i3 (char3), index i4 (num2));

29

Data Shape: Average Rowscreate table averagerows ( newid bigint auto_increment primary key, real_id binary(20), num0 bigint, char1 varchar(60), num1 decimal (10,2), num2 double, char2 varchar(60), char3 varchar(50), index id_helper (real_id(6)), index i1 (num0), index i2 (char1), index i3 (char3), index i4 (num2));

30

Data Shape: Average Rows

create trigger averagerows_bi before insert on averagerows for each row begin select count(*) into @c from averagerows where real_id = new.real_id; if @c != 0 then set @m=concat('Duplicate entry ''',hex(new.real_id),''' for key ''PRIMARY'''); signal SQLSTATE '23000' set message_text=@m, mysql_errno=1062; end if; end;

31

Data Shape: Average Rows

create trigger averagerows_bi before insert on averagerows for each row begin select count(*) into @c from averagerows where real_id = new.real_id; if @c != 0 then set @m=concat('Duplicate entry ''',hex(new.real_id),''' for key ''PRIMARY'''); signal SQLSTATE '23000' set message_text=@m, mysql_errno=1062; end if; end;

Do we have this one already?YES

Raise an error

32

Data Shape: Large Rows

create table largerows_ai ( id bigint primary key, payload1 varchar(2000), payload2 varchar(2000), payload3 varchar(2000), payload4 varchar(2000));

33


create table largerows_ai ( id bigint auto_increment primary key, payload1 varchar(2000), payload2 varchar(2000), payload3 varchar(2000), payload4 varchar(2000));

34


create table largerows_ai ( id bigint auto_increment primary key, payload1 varchar(2000), payload2 varchar(2000), payload3 varchar(2000), payload4 varchar(2000), index i1 (payload1(40)), index i2 (payload2(767));

35

Storage Engines

36

Storage Engine: InnoDB

Very Mature

Well understood

Continually updated

Supported by Galera Cluster

37

Storage Engine: TokuDB Was separately-licensed; bought by Percona last year

Built into Percona Server now

Extreme data compression

Online table alters

Other benefits

Not supported by Galera Cluster

38

Storage Engine: Deep Separately-licensed product

Distributed as a complete package, based on 5.6

Does data compression

Self-tuning using constant data analysis

Other benefits

Not supported by Galera Cluster

39

Filesystem

ext4, xfs, others

Journaling uses IO and can be disabled

40

T E S T I N G M E T H O D S

41

HardwareLoad Generator Database

1 Gbe

Dell 2950-III 32 GB RAM Two 4-core 3GHz Xeon processors SSD and HDD storage Ubuntu 14.04 Measured SSD IOPS (fio: random RW / 5G file): 9750 read, 3254 write

Desktop 8 GB RAM Intel i7 processor SSD storage Ubuntu 14.04

42

SoftwareLoad Generator Database

Ubuntu 14.04 Percona Server 5.6.28 deepSQL 5.6.28-21214

Ubuntu 14.04 go v.1.2.1

43

Data Loading in go for speed

Loop until stopped: Fill buffer with 1000 (or 10000) rows Send buffer to LOAD DATA INFILE

Very low CPU and network on load-gen box

Concurrency of 1 and 20

github.com/bobburgess

http://github.com/bobburgess

44

Data Usage (Work that DB) in go for speed

Loop until stopped: Select by PK Select by each secondary index

(column starts with random value) Insert a row of random data

Very low CPU and network on load-gen box

Concurrency of 20, 60, and 100


http://github.com/bobburgess

45

DB Server Measurements

in bash cause Im lazy (and speed not required)

Loop until stopped: Take measurements Output measurements Remember old values (for differential measurements) sleep for 1 sec minus the time taken for the measurements

Very lightweight


46


Rows per second inserted & selected

Disk IOs per second, read & write

Bytes per second, read & write

Size of mysqld process

Size of data on disk

CPU %, User-time & System-time

47


Rows per second inserted & selected

Disk IOs per second, read & write

Bytes per second, read & write

Size of mysqld process

Size of data on disk

CPU %, User-time & System-time

Measure of throughput

Possible to use slower storage?

Possible to use slower storage?

Wasteful of memory? Bloated?

Possible to use less storage?

Lower = can handle more load

48

Testing Flow

Start DB monitor on DB server (output to file)

Start up required processes on load-gen box

Check DB server for unexpected activity

Check load-gen box for unexpected activity

Check load-gen box for CPU or network bottlenecks

Stop test processes when data set large enough

49

Plotting the Data

R

Define columns columns

50

Plotting the Data Example: Two plots with running-average lines

par(mar=c(5, 9, 4, 6) + 0.1)options(scipen=15)Xmin=1Xmax=14432Ymin=0Ymax=120smooth=100# Firstplot(seq(Xmax), avg_ai_i_load_20ses_ssd_inno$IOPSW[Xmin:Xmax], col="red", pch=".", xlab="", ylab="", ylim=c(Ymin,Ymax), type="p", axes=FALSE, main="")axis(2,ylim=c(Ymin,Ymax), col="red", las=1)mtext("IOPS - Write", side=2, line=3)box()lines(seq(Xmax), filter(avg_ai_i_load_20ses_ssd_inno$IOPSW[Xmin:Xmax], rep(1/(smooth+1),(smooth+1)), sides=2), col="red", lwd=2)par(new=TRUE)# Secondplot(seq(Xmax), avg_rnd_i_load_20ses_ssd_inno$IOPSW[Xmin:Xmax], col="blue", pch=".", xlab="", ylab="", ylim=c(Ymin,Ymax), type="p", axes=FALSE)lines(seq(Xmax), filter(avg_rnd_i_load_20ses_ssd_inno$IOPSW[Xmin:Xmax], rep(1/(smooth+1),(smooth+1)), sides=2), col="blue", lwd=2)# Finish upaxis(1,at=seq(0,Xmax,60),labels=seq(0,Xmax/60))mtext("Minutes", side=1, col="black", line=2)legend("topright",c("Auto-Inc PK","Random PK"),fill=c("red","blue"))

51

R E S U LT S

InnoDB

Storage Engine Row Size________ Primary Key_________ Indexing Sessions InnoDB TokuDB Tiny Average Large Auto-Inc Auto-Inc+Trig PK Only 1 20 Deep Random Random Lrg Multiple

56


57


58


59


60


61


62


200,000

63


64


65


66


67


68


69


70


71


72


73



82


83


84


85


?!


93


94

95

InnoDBBulk-Load

Single-Session 1 Multi-Session 0

Primary Key

Auto-Inc 1 Random 1 Large 0

TokuDB

96

97



98


99

100


101


102


103


40-60

104


105


106


107


108


109


110


Difference with InnoDB is not as great

111


112


113


114


115


116


117


118


119


120


121


122


123


124


125


126


127


128


129


130


131


132


133


134


135


136


137


138


139



Same shape as PK-Only, 1 Session

146


147


148


deepSQL

153

154



InnoDB

TokuDB

157


158


161


162


163


164


165


166


167


168


171


172


173


174


175


176


177


178


179


180


181


182


183


184


185


186


InnoDB

TokuDB

194


195


196


197


198


199


200


201


202


203


204


205

Usage Performance

206

Usage Performance Each row size

Auto-Increment PK

Inserted by multiple threads

disk files of 2X or 3X cache size

207

Usage Performance

Loop this block: Several selects One Insert

20, 60, 100, 200 concurrent

Count Com_insert and Com_select on server side

208

Usage Performance

Average Rows InnoDB TokuDB Deep

Rows 3,264,053 6,742,470 5,324,183

Avg. Row Length 366 174 180

~2 GB on disk

209

Tiny Rows (Select)

20 sessions 60 sessions 100 sessions 200 sessions

210

Tiny Rows (Insert)

211

Average Rows (Select)

212

Average Rows (Insert)

213

Large Rows (Select)

214

Large Rows (Insert)

215

N O M O R E C H A R T S !

216

What I Learned

Vast differences depending on use-case

Some patterns not as bad as first thought (i.e. Random PK)

Some patterns worse than first thought (i.e. with bytes/row)

217

Take-Aways

Test for your use-case

Dont assume you already know how the DB will respond

Dont trust anyone, especially me. Test it yourself!

218

T H A N K Y O U !S L I D E S W I L L B E O N L I N E

B O B . B U R G E S S @ S H O P I F Y. C O M G I T H U B . C O M / B O B B U R G E S S

P L E A S E F I L L O U T A N E VA L U AT I O N !

is hiring!

Data & Analytics

Storage Methods for Nonstandard Data Patterns