Postgres-BDR with Google Cloud Platform

POSTGRES-BDR WITH GOOGLE CLOUD PLATFORM

ROCKPLACE

Email: [email protected]://rockplace.co.kr

Copyright 2016 Rockplace Inc.. All right Reⓒ -served

Copyright All right Reserved by 2016 Rockplace Inc.ⓒ

윤성재 ( 공작명왕 )[email protected] https://gongjak.me

• 리눅스 엔지니어• 서비스 운영 Administrator• Database 운영 Administrator • 클라우드 솔루션 아키텍쳐

• 1984 년 Apple II 와 만남• 1994 년 Linux 와의 첫 만남• 2015 년 서비스에 PostgreSQL 처음 적용• 2016 년 빠르게 훑어보는 구글 클라우드 플랫폼

Copyright All right Reserved by 2016 Rockplace Inc.ⓒ

AGENDA구글 클라우드 플랫폼- 테스트 환경으로 선택한 이유

Postgres-BDR- Installation & Setup- Create Cluster

pgbench Test- Test 실행- 결과

구글 클라우드 플랫폼테스트 환경으로 선택한 이유

OpenSource PLACE, ROCKPLACE

Copyright 2016 Rockplace Inc. All rights Reservedⓒ

Create Instance on Google Cloud Platform

• Region 위치 : asia-east, eu-west, us-central

• 각 Region 별로 3 대씩 , 총 9 대 생성

• Region 별 2 대는 BDR 설치 , 1 대는 pgbench 테스트용

• OS : Debian 8 Jessie



VPN 을 이용한 Network 구성

Separated NetworkUsing VPNSetup Route table



GCP 의 Network 구성

Single NetworkNo VPNNo Setup Route table

POSTGRES-BDRInstallation & Setup



Install Postgres-BDRBDR extension 이 반드시 설치되어야 한다 . (package 의 경우 , postgresql-bdr-contrib)

sudo sh -c 'echo "deb [arch=amd64] http://packages.2ndquadrant.com/bdr/apt/ jessie-2ndquadrant main" >> /etc/apt/sources.list.d/2ndquad-rant.list'

wget --quiet -O - http://packages.2ndquadrant.com/bdr/apt/AA7A6805.asc | sudo apt-key add -

sudo apt-get update

sudo apt-get -y install postgresql-bdr-9.4-bdr-plugin



Initial PostgreSQLPostgreSQL 에 문제가 있다고 생각되어 초기화를 하고자 할 때

sudo service postgresql stopsudo rm -rf /var/lib/postgresql/9.4sudo rm -rf /etc/postgresql/9.4/mainsudo rm -rf /service/db/pgsql/9.4/mainsudo chown -R postgres.postgres /var/lib/postgresqlsudo pg_createcluster -d /var/lib/postgresql/9.4/main 9.4 mainsudo service postgresql start



Add DB accountUsername : dbadmin Password : dbadmin1234

sudo su - postgres

psql -dpostgres -c "CREATE ROLE dbadmin LOGIN PASSWORD 'dbadmin1234' superuser;"



BDR 을 위한 PostgreSQL 설정/etc/postgresql/9.4/main/postgresql.conf

• listen_addresses = '*'

• shared_preload_libraries = 'bdr'

• wal_level = 'logical'

• track_commit_timestamp = on

• max_connections = 100

• max_wal_senders = 10

• max_replication_slots = 10

• max_worker_processes = 10



BDR 을 위한 PostgreSQL 설정/etc/postgresql/9.4/main/pg_hba.conf

all all 10.128.0.0/9 md5# The standby server must connect with a user that has replication privi-leges.# for BDR settingecho 'host replication postgres 10.128.0.0/9 trust

# 중간 IPv4 부분을 아래와 같이 10.128.0.0/9 해 줘야 로컬에서도 접속이 됨host all all 10.128.0.0/9 trust

POSTGRES-BDRCreate Cluster



Setup Postgres-BDRAll nodes

sudo su – postgrescreatedb testdbpsqlpostgres=# \connect testdbYou are now connected to database "testdb" as user "postgres".

testdb=# CREATE EXTENSION btree_gist;CREATE EXTENSION

testdb=# CREATE EXTENSION bdr;CREATE EXTENSION



Setup Postgres-BDRInstance 생성 순서 및 Cluster Join

2. bdr-asia-2

1. bdr-asia-1 3. bdr-us-1

4. bdr-us-2

5. bdr-eu-1

6. bdr-eu-2



Setup Postgres-BDRBDR Cluster 내부 구성

bdr-asia-2

bdr-asia-1 bdr-us-1

bdr-us-2

bdr-eu-1

bdr-eu-2



Setup Postgres-BDRbdr-asia-1 서버에서의 설정 ( 메인 노드 그룹 생성 )

testdb=# SELECT bdr.bdr_group_create(local_node_name := 'asia-node-001',node_external_dsn := 'host=bdr-asia-1 port=5432 dbname=testdb’); bdr_group_join----------------

(1 row)



Setup Postgres-BDRbdr-asia-1 서버에서의 설정 ( 확인 )

testdb=# SELECT bdr.bdr_node_join_wait_for_ready(); bdr_node_join_wait_for_ready------------------------------

(1 row)



Setup Postgres-BDRbdr-asia-2 서버와 그 외 서버에서의 설정 ( 메인 그룹 참여 )

testdb=# SELECT bdr.bdr_group_join(local_node_name := 'asia-node-002',node_external_dsn := 'host=bdr-asia-2 port=5432 dbname=testdb',join_using_dsn := 'host=bdr-asia-1 port=5432 dbname=testdb'); bdr_group_join----------------

(1 row)



Setup Postgres-BDRbdr-asia-2 서버와 그 외 서버에서의 설정 ( 확인 )

testdb=# SELECT bdr.bdr_node_join_wait_for_ready(); bdr_node_join_wait_for_ready------------------------------

(1 row)



Add node

• pg_basebackup 으로 SOURCE 로부터 전체 백업• bdr_init_copy 명령어를 이용

• 주요 Configuration Setting : 충분히 크게 ~~max_wal_senders = 10max_replication_slots = 10max_worker_processes = 10

• bdr_init_copy_postgres.log 를 확인



Remove node

• 함수로 쉽게 노드를 제거 할 수 있다고 하나 , 실제로는 함수 실행 후 확인해보면 노드 정보가 남아있다 . (select * from bdr.bdr_nodes;)

• node 하나만 제거

• 여러개의 노드를 한 번에 제거

• asia-node-003 제거

SELECT bdr.bdr_part_by_node_names(ARRAY['node-1']);

SELECT bdr.bdr_part_by_node_names(ARRAY['node-1', 'node-2', 'node-3']);

SELECT bdr.bdr_part_by_node_names(ARRAY['asia-node-003']);



Monitoring

• Monitoring nodes

• Monitoring connected peers using pg_stat_replication

• Monitoring replication slots

• Monitoring global DDL locks

SELECT * FROM bdr.bdr_nodes;

SELECT * FROM pg_stat_replication;SELECT pg_xlog_location_diff(pg_current_xlog_insert_location(), flush_location) AS lag_bytes, pid, application_nameFROM pg_stat_replication;

SELECT * FROM pg_replication_slots;SELECT slot_name, database, active, pg_xlog_location_diff(pg_current_xlog_insert_location(), restart_lsn) AS re-tained_bytesFROM pg_replication_slotsWHERE plugin = 'bdr';

select * from bdr.bdr_global_locks ;



Backup

• 백업은 snapshot 또는 pg_dump, pg_basebackup 이용

PGBENCH TESTTest 실행



pgbench Test 실행pgbench -U postgres -P 5432 -i testdb

NOTICE: table "pgbench_history" does not exist, skippingNOTICE: table "pgbench_tellers" does not exist, skippingNOTICE: table "pgbench_accounts" does not exist, skippingNOTICE: table "pgbench_branches" does not exist, skippingcreating tables...100000 of 100000 tuples (100%) done (elapsed 0.15 s, remaining 0.00 s).vacuum...set primary keys...done. -U postgres : postgres 유저로 접속

-P 5432 : 5432 포트로 접속 -i : 테스트준비 testdb : 디비로 testdb 를 이용하겠다 .



pgbench Test 실행pgbench -U postgres -p 5432 -S -c 10 -t 10000 testdb

starting vacuum...end.transaction type: SELECT onlyscaling factor: 1query mode: simplenumber of clients: 10number of threads: 1number of transactions per client: 10000number of transactions actually processed: 100000/100000latency average: 0.000 mstps = 11496.597122 (including connections establishing)tps = 11520.436044 (excluding connections establishing)

-U postgres : postgres 유저로 접속 -P 5432 : 5432 포트로 접속 -i : 테스트준비 testdb : 디비로 testdb 를 이용하겠다 .



Custom query 를 이용한 pgbench test

• create table tbl_bdr(c1 int);

• vi test.sqlinsert into tbl_bdr values (3);

• pgbench -U postgres -n -S -T 60 -c 10 -f test.sql testdb

transaction type: Custom queryscaling factor: 1query mode: simplenumber of clients: 10number of threads: 1duration: 60 snumber of transactions actually processed: 379920latency average: 1.579 mstps = 6331.344284 (including connections es-tablishing)tps = 6332.955115 (excluding connections es-tablishing)

PGBENCH결과



각 Region 별 ping 테스트 결과Count : 100

Ping 방향 결과Asia US 100 packets transmitted, 100 received, 0% packet loss, time

99131msrtt min/avg/max/mdev = 153.147/153.280/154.927/0.547 ms

US Asia 100 packets transmitted, 100 received, 0% packet loss, time 99132msrtt min/avg/max/mdev = 153.192/153.291/153.849/0.403 ms

Asia EU 100 packets transmitted, 100 received, 0% packet loss, time 99134msrtt min/avg/max/mdev = 257.257/257.393/258.276/0.412 ms

EU Asia 100 packets transmitted, 100 received, 0% packet loss, time 99135msrtt min/avg/max/mdev = 257.297/257.418/258.220/0.582 ms

US EU 100 packets transmitted, 100 received, 0% packet loss, time 99175msrtt min/avg/max/mdev = 105.583/105.661/106.485/0.435 ms

EU US 100 packets transmitted, 100 received, 0% packet loss, time 99075msrtt min/avg/max/mdev = 105.564/105.661/106.434/0.132 ms



case 1. 한 대의 서버에만 데이타 insert

Num-ber of Client

Time (s)

number of transactions ac-tually processed

latency average

tps (a) tps (b)

1 600 608506 0.986 ms 1014.175220 1014.1866075 600 2042736 1.469 ms 3404.536435 3404.75797510 600 3928831 1.527 ms 6547.974117 6548.823522

* tps (a) : including connections establishing* tps (b) : excluding connections establishing



Target : 1 Server / single table / 1 Client / Time 600s







8:38:438:45:088:45:37



case 3-1. 모든 6 대 서버의 동일한 table 에 데이타 insert1 Client


Server number of transac-tions actually pro-cessed

latency average

tps (a) tps (b)

asia-1 534279 1.123 ms 890.464274 890.475745asia-2 502395 1.194 ms 837.324570 837.335128us-1 599298 1.001 ms 998.827496 998.839511us-1 579556 1.035 ms 965.924930 965.935587eu-1 536775 1.118 ms 894.624563 894.635521eu-2 491666 1.220 ms 819.441429 819.451690



Target : All Server / single table / 1 Client / Time 600s



case 2. 모든 6 대 서버의 동일한 table 에 데이타 insert5 Client



latency average

tps (a) tps (b)







case 2-3. 모든 6 대 서버의 동일한 table 에 데이타 insert10 Client



latency average

tps (a) tps (b)







case 3-1. 모든 6 대 서버의 서로 다른 table 에 데이타 insert1 Client



latency average

tps (a) tps (b)




Target : All Server / each table / 1 Client / Time 600s






latency average

tps (a) tps (b)










latency average

tps (a) tps (b)







마치며…

• Support Multi-Master

• Multi Region Replication

• Effect Network latency

• Community Support Email : [email protected] : https://groups.google.com/a/2ndquadrant.com/forum/#!forum/bdr-list .

mailto:[email protected]

https://groups.google.com/a/2ndquadrant.com/forum/#!forum/bdr-list





Q & A

감사합니다 .http://gongjak.me

[email protected]

Software

Postgres-BDR with Google Cloud Platform