32
© 2016 Internet Initiative Japan Inc.

IIJにおけるGlusterFS利用事例 GlusterFSの詳解と2年間の運用ノウハウ

  • Upload
    iij

  • View
    165

  • Download
    0

Embed Size (px)

Citation preview

© 2016 Internet Initiative Japan Inc.

© 2016 Internet Initiative Japan Inc.

© 2016 Internet Initiative Japan Inc.

© 2016 Internet Initiative Japan Inc.

© 2016 Internet Initiative Japan Inc.

© 2016 Internet Initiative Japan Inc.

Number of Bricks: 1 x (2 + 1) = 3

Redundancy

© 2016 Internet Initiative Japan Inc.

© 2016 Internet Initiative Japan Inc.

© 2016 Internet Initiative Japan Inc.

© 2016 Internet Initiative Japan Inc.

© 2016 Internet Initiative Japan Inc.

© 2016 Internet Initiative Japan Inc.

© 2016 Internet Initiative Japan Inc.

© 2016 Internet Initiative Japan Inc.

# gluster volume get <Volume名> all

struct volume_options options[] = { ... { .key = {"page-size"}, .type = GF_OPTION_TYPE_SIZET, .min = 4096, .max = 1048576 * 64, .default_value = "131072", .description = "Page size with which read-ahead performs server I/O” }

© 2016 Internet Initiative Japan Inc.

• •

.(mount point) foo/ sample.txt

.(brick dir) .glusterfs 47/ 03/ 4703ed86-72b5-4573-997c-7e2339e2b1d7 foo/ sample.txt

# getfattr -d -e hex -m . foo/sample.txt # file: foo/sample.txt trusted.bit-rot.version=0x020000000000000057a08889000abad6 trusted.ec.config=0x0000080301000200 trusted.ec.size=0x0000000000000007 trusted.ec.version=0x00000000000000010000000000000001 trusted.gfid=0x4703ed8672b54573997c7e2339e2b17d7

© 2016 Internet Initiative Japan Inc.

© 2016 Internet Initiative Japan Inc.

–•

–•

–•

© 2016 Internet Initiative Japan Inc.

© 2016 Internet Initiative Japan Inc.

1. d2r2 をFUSEマウント

GlusterFSのクラスタならどこでも良い

2. 書き込み

3. DHT(分散ハッシュテーブル) から書き込み先を決定

4. Brickに file1 を書き込み

© 2016 Internet Initiative Japan Inc.

Brick1 Brick2 Brick3/dir1 0 - 99 100 - 199 200 - 299/dir2 100 - 199 400 – 499 300 - 399/dir3 500 - 599 200 - 299 100 - 199

© 2016 Internet Initiative Japan Inc.

© 2016 Internet Initiative Japan Inc.

© 2016 Internet Initiative Japan Inc.

© 2016 Internet Initiative Japan Inc.

»

© 2016 Internet Initiative Japan Inc.

echo "deadline" > /sys/block/<block name>/queue/scheduler echo "256" > /sys/block/<block name>/queue/nr_requests echo "16" > /proc/sys/vm/page-cluster sysctl -w vm.swappiness=10 sysctl -w vm.dirty_background_ratio=1 sysctl -w kernel.sched_wakeup_granularity_ns=15 sysctl –w vm.vfs_cache_pressure=50

PRUNEFS = “fuse.glusterfs” PRUNEPATHS = “<Brick Directory>”

© 2016 Internet Initiative Japan Inc.

gluster volume set <VOLNAME> cluster.entry-self-heal off

mount –t xfs –o inode64,noatime,nodiratime,attr2,delaylog,noquota

© 2016 Internet Initiative Japan Inc.

© 2016 Internet Initiative Japan Inc.

© 2016 Internet Initiative Japan Inc.

© 2016 Internet Initiative Japan Inc.

© 2016 Internet Initiative Japan Inc.