Business’Con*nuity’with’Docker · Architecture’for’DRwith’ Docker ... Reference:’hdp://aws.clouddesignpaern.org /index.php/ MURORAN INSTITUTE OF TECHNOLOGY

MURORAN INSTITUTE OF TECHNOLOGY

Business Con*nuity with Docker

2015/06/04

Yoshitaka Kuwata Muroran Ins*tute of Technology

CloudOpen Japan 2015

Copyright(C) 2015 by Yoshitaka Kuwata


Overview of Talk 1.  Who is Talking 2.  Mo*va*on of Disaster Recovery 3.  Exis*ng Solu*ons 4.  Requirement of DR 5.  Architecture for DR with Docker 6.  Evalua*ons 7.  Discussions 8.  Conclusions

Copyright(C) 2015 by Yoshitaka Kuwata 2


Yoshitaka Kuwata Associate Professor Center for Mul*media Aided Educa*on Muroran Ins*tute of Technology, Hokkaido • Educa*on of Basic IT course • Design and opera*on of campus network & computer systems • Research on Cloud Architecture and OSS Project Analysis

1. Who am I

MURORAN: 1.5 hours from

Sapporo, One hour from Chitose

Airport.



2. Mo*va*on of Disaster Recovery


Awareness of Disaster

Preven*on

Business Con*nuity

Management

Design of Exis*ng Systems

Compliance

Lessons of Great

Earthquakes

Interna*onal Standard

・Func*onal Design ・Non Func*onal Design 　Capacity 　Extensibility 　Operability 　Maintainability 　Security 　Fault Tolerance 　　Reliability 　　Availability 　

・Analysis 　Iden*fy Cri*cal Factors ・Management System 　Business Impact Analysis Establish BCP Documenta*on ・Implementa*on 　Apply BCP 　Training ・Opera*on

Need to apply BCP to exis*ng systems.

The design should also be used for next systems.


3. Requirement form light-‐weight DR

Pre Condi*ons •  Primary systems are located on local site. (not all of systems are cloud ready) •  Cloud services are used only for backup data and backup system. •  Make a set of daily backup copy of data to the clouds. •  When the primary systems downs, backup systems are started manually. •  When recovered, manually copy data from backup systems to primary

systems, then stop backup systems.


Requirements (1) Data must be kept in safe in remote loca*ons (2) Keep the service running in remote site, when primary site is down

(3) Switch loca*on of service manually. (4) The architecture should be the same with current design.


4. Exis*ng Solu*ons in Cloud

No Name Method

1 Mul*-‐Datacenter In order to avoid failure of a datacenter, use mul*ple servers in different datacenters located in different regions.

2 Sorry Page When main site is down, redirect requests to backup servers; sorry server

3 Cross-‐Region Replica*on

Make a set of backup copy to different loca*on.


AWS Cloud Design Padern(AWS-‐CDP） for Disaster Recovery

Reference: hdp://aws.clouddesignpadern.org/index.php/


4. Mul*-‐datacenter in AWS-‐CDP

7

Servers A

EC2 instances EC2 instances

dispatch Region A Region B


Load balancer

Servers B



4. Sorry page in AWS-‐CDP

8

Primary Servers

S3

EC2 instances Object Storage System

•  Health check •  Change DNS entry


DNS Server（Route 53）

Backup Server



4. Cross-‐Region Replica*on in AWS-‐CDP

9

Primary Server Secondary Server

EC2 instance EC2instance

DB DB Backup Restore

Remote Copy

Region A Region B


Note: This padern is one of AWS-‐CDP candidates.



5. DR with Docker Useful Features of Docker

(1) Light-weight Virtualization with LXC 　Partitioning of OS(2) Differential File System AUFS, LVM thin provisioning, Overlayfs. (3) Portable Image Docker Image (4) Repository Management 　Docker-HUB, Local Repository(5) Building Automation of Docker Image Dockerfile (6) Optimal Runtime Environment Minimum set of programs are deployed in images



5. How we make use of Docker for DR

We focused on two aspects of Docker •  Differential File System

–  We can reduce the size of daily backup volume

•  Portability of Images –  We can build private repository on academic cloud service, and commercial service as well.

–  We can run secondary site on cloud services.

•  Key Technologies –  Design paderns of backup systems –  Implementa*on and use of private repository –  Security

11 Copyright(C) 2015 by Yoshitaka Kuwata


Architecture for DR

LMS

Data

Storage System

Data

①Make Backup

LMS

② run alterna*ve server

Data Container (Docker)

LMS：Learning Management System

12

University

Academic Cloud AWS

Storage System

Data

LMS

Data

AWS can be used as alterna*ve of

Academic Cloud

Campus LAN

SINET

③Recovery



5. Architecture for Disaster Recovery with Docker

13

Primary System

A

A

Docker Repository

Servers on site

Servers on Cloud Service

pull & run

push

Docker Engine

Docker Engine

B

B

Linux

App1

App2

A

B


1.  Build Private Docker Repository on Cloud Service •  The Repository can be built with backend

2.  Copy Backup data of Primary System to a Docker Repository on Cloud Services

Overview

Secondary system


5. Make a set of backup copy to repo.


1.  Push images to local repository on Cloud Services.

Daily Opera*ons Primary System

A

Docker Repository

Servers on site


Docker Engine

B

Linux

App1

App2

A

B


5. Switching to Secondary system


1.  Launch secondary system from image in Docker Repository on Cloud Service

2.  Redirect access to Secondary system

Switching to Secondary system Primary System

Secondary system

A

A

Docker Repository

Servers on site


pull & run

Docker Engine

Docker Engine

B

B

Linux

App1

App2

A

B

OFFLINE


5. Recovery Opera*on


1.  Push data of Secondary System to Docker Repository 2.  Stop Secondary System 3.  Launch Primary system with data from Docker

Repository 4.  Redirect access to Primary System

Primary System

A

A

Docker Repository

Servers on site


push

Docker Engine

Docker Engine

B

B

Linux

App1

App2

A

B

Secondary system


6. Evalua*on Environment Item Specifications Application Moodle 2.7 with Language Pack (Japanese) Number of Course 0〜20 Number of Sessions 15 / course Repository docker-‐registry on Docker in a local machine CPU Intel Core2 Quad Q8400 2.66GHz Memory 4GB Disks SATA 128GB SSD NIC 1000Base-‐T (1Gbps) OS Ubuntu 14.04.1LTS Server Docker V1.2.0, build fa7b24f


moodle

Docker Repository

Primary System

push

pull & run

pull & run

push Docker Engine

Secondary System

moodle


6. Seing of Moodle •  Seing 15 Sessions for each classes

–  Plase course material for each session •  Allocate Numbers of classes and check the required

resources.



6. Results : overview

•  Image size increases with every entry of courses – Only the difference is stored in the images

•  Time to push repositories depends on the image size


Linux moodle Japanese locale Course 1 Course 2 Course N

≒1GB 32MB 32MB


6. Results: number of course VS image size


Image Size

Time to put repo.


7. Discussions


•  Limita*on of snapshot on Docker – Docker can have only 127 level of snapshot

# docker build –t test . Sending build context to Docker daemon 4.096 kB Sending build context to Docker daemon Step 0 : FROM ubuntu -‐-‐-‐> 5ba9dab47459 Step 1 : RUN date -‐-‐-‐> Running in b4849b849f31 Tue Feb 17 09:59:37 UTC 2015 -‐-‐-‐> 36904b4e1c8d Removing intermediate container b4849b849f31 Step 2 : RUN date -‐-‐-‐> Running in ae5abb842961 Tue Feb 17 09:59:42 UTC 2015 -‐-‐-‐> 7c94c175a4b6 Removing intermediate container ae5abb842961 Step 3 : RUN date -‐-‐-‐> Running in f107b4d4ec6e Tue Feb 17 09:59:44 UTC 2015 -‐-‐-‐> 95dd7644aeca Removing intermediate container f107b4d4ec6e : : : Step 121 : RUN date INFO[0296] Cannot create container with more than 127 parents

This is not the limita*on of file system but the limita/on of Docker.

i.e. happens with AUFS, Overlay FS, and LVM


7. Discussions (cont.) •  Work around for 127-‐parent limit (1)  Merge layers with export and import


# container_id=$(docker run -‐d test2 /bin/bash -‐c "") # docker export $container_id | docker import -‐ test2 34e82f0c70802fed06348fd805628ffc9d53374939b82fc2fb2cdcdec14f34de # docker images -‐-‐tree Warning: '-‐-‐tree' is deprecated, it will be removed soon. See usage. ├─34e82f0c7080 Virtual Size: 188.1 MB Tags: test2:latest └─511136ea3c5a Virtual Size: 0 B └─27d47432a69b Virtual Size: 192.5 MB └─5f92234dcf1e Virtual Size: 192.7 MB └─51a9c7c1f8bb Virtual Size: 192.7 MB └─5ba9dab47459 Virtual Size: 192.7 MB Tags: ubuntu:latest


7. Discussions (cont.) •  Work around for 127-‐parent limit (2) Take a snapshot and con*nue the image


Moodle base:V1

Image Instance

Moodle base:V1

Moodle base:V1.1

Moodle base:V1.1.1

run

Stop & commit

Moodle base:V1.1 run

Stop & commit

Moodle base:V1.1.1 run

Stop & commit Moodle base:V1.1.1.1


7. Discussions (cont.) •  Work around for 127-‐parent limit (2) Take a snapshot and con*nue the image


Moodle base:V1

Image Instance

Moodle base:V1

Moodle base:V1.1

Moodle base:V1.2

run

Stop & commit

Moodle base:V1.1

start

Stop & commit

Moodle base:V1.3

Moodle base:V1.2 Stop & commit

start

start


8. Conclusions


•  Light-‐weight DR architecture with Docker is proposed

•  Prototype system is built and evaluated – Func*ons of the system are verified – Limita*on of Docker was found and work-‐around is proposed.

• 

Documents

Business’Con*nuity’with’Docker · Architecture’for’DRwith’ Docker ... Reference:’hdp://aws.clouddesignpaern.org /index.php/ MURORAN INSTITUTE OF TECHNOLOGY