Dr관련 세미나 자료 v2333

Copyright © 2014 Juniper Networks, Inc. 1 Copyright © 2014 Juniper Networks, Inc.

DISASTER RECOVERY AND INTER-DC WORKLOAD MOBILITY

APRIL [email protected]

Copyright © 2014 Juniper Networks, Inc. 2

연결된 세상의 힘CONNECT EVERYTHING. EMPOWER EVERYONE.


AGENDA

Business and technologies requirement

Storage networking DCI design considerations

Disaster Recovery and Workload Mobility DCI design considerations

Juniper Data Center Connectivity Solution


DISASTER RECOVERY DCI 솔루션 : 사업적 요구 사항

데이터 가용성• Client to Storage• Server to Storage• Storage to Storage• Measured by RTO and RPO

• Different backup/replication methods give different RTO/RPO as generally as they go down the cost of the method goes up

재난 복구

사업 연속성• One of the critical function of over all BC planning

정부 규제 • Storage availability and continuity of access

• http://www.drj.com/tools/tools/dr-rules-and-regulations.html• HIPAA, SoX

민첩성• Leveraging DR sites for workload balancing

http://www.drj.com/tools/tools/dr-rules-and-regulations.html


DISASTER RECOVERY DCI 솔루션 : 기술적 요구사항

IT 서비스의 성공적인 재개를 위한 필요 사항• 데이터의 가용성

• Consistent data availability • 어플리케이션 / 컴퓨팅 자원의 가용성

• Sufficient computing power and application software• 네트워크 가용성

• For compute, data resource communication and user connections• 데이터 세션 리다이렉션

• User’s request must be redirected to available application compute resources• Data access requests must be redirected to available data store

네트워크 요구 사항• 블록 디스크 엑세스 , 복제 타입 , 거리 등에서 오는 고려 사항

• Low Latency• High B/W• No loss/low loss• Reliable/Resilient


DISASTER RECOVERY DCI 솔루션 : 사업적 요구 사항

복구 목표 시간 (Recovery Time Objective)

복구 목표 지점 (Recovery Point Objective)Uptime Uptime 년간 최대 허용

다운 시간

Five nines 100.00% 5 분 35 초

Four nines 99.99% 52 분 33 초

Three nines 99.90% 8 시간 46 분

Two nines 99.00% 87 시간 36 분

One nine 90.00% 36 일 12 시간

가용성 테이블

“Zero-time” / “Zero-loss” RTO/RPO

높은 RTO 및 RPO 기준은 더 잘 디자인된 DCI 솔루션을

필요로함 .

DECLARE DISASTER10 a.m.

RPO RTO

RPO: Amount of data lost from failure, measured as the amount of time from a disaster event

RTO: Targeted amount of time to restart a business service after a disaster event

5a.m.

6a.m.

7a.m.

8a.m.

9a.m.

10a.m.

11a.m.

12a.m.

1p.m.

2p.m.

3p.m.

4p.m.

5p.m.

6p.m.

7p.m.


AGENDA






DATA CENTER 들 사이의 데이터 가용성

데이터 복제 기술들

• 동기식 복제 (Synchronous replication)Any write request is acknowledged only after replication to target is acknowledged

• 비동기식 복제 (Asynchronous replication)A write request is acknowledged as soon as write is complete at local data store, replication request is separately handled

• 스냅샷 복제 (Snapshot replication)Similar to asynchronous replication, however the replication requests are handled in batches


세가지 데이터 복제 기술들

동기식 복제 비동기식 복제 스냅샷 복제

• Any write request is acknowledged only after replication to target is acknowledged

• "Zero-time” / “Zero-loss" RTO/RPO Mostly across MAN distances

• Low Latency• Guaranteed Bandwidth

A write request is acknowledged as soon as write is complete at local data store, replication request is separately handled

WAN Higher Average throughput

Similar to asynchronous replication, however the replication requests are handled in batches

WAN Higher Average throughput


복제 네트워크 영향

가용한 대역폭이 충분한 시간 동안 제공되지 못한다면 , 어플리케이션은 손상되고 복구를 위한 사용자의 개입이 필요 할 수 있음 .

동기식 복제에서 필요 대역폭이 가용 대역폭 이상일 경우 , 어플리케이션의 성능은 레이턴시의 증가로 인해 많이 떨어 질 수 있음 .

비동기식 복제는 어플리케이션 성능의 영향 없이 Data Rate을 부드럽게 허용할 수 있음 .

비동기식 복제는 변화를 모아 전달 하기 때문에 필요한 대역폭을 줄일 수 있음 .


복제에 대한 접근 및 프로토콜

원거리 전송 타입 (FC 또는 SAN 확장 ) FC (MAN distance only) iSCSI (WAN distance), iSCSI really more for Initiator to target. i.e. client/server system to storage. FCIP (WAN distance), FCIP predominantly used for bridging FC SANs over long distances iFCP, xWDM

모든 SCSI 의 전송 방법은 기기 사이에서 IP 네트워크를 통해 명령

호스트 기반 복제 어레이 기반 복제


ISCSI STACK 이 FCIP STACK 보다 효율적

ISCSI PDU 는 ISCSI CDB (Control Descriptor Blocks) 옵션과 함께 세션 명령과 데이터를 포함 iSCSI 는 3260 포트 사용 데이터 및 헤더 무결성을 위한 에러 체크 기능 IPSEC 기능을 통한 보안 세션 및 데이터 사용

FCIP PDU 는 FCIP 세션 제어 정보를 포함 FCIP 는 3225 포트 사용

Data LinkIP

TCPFCIP

Data LinkIP

TCPiSCSI

Physical Physical

SCSI FCPSCSI


ISCSI DCI 적용이 보다 단순

FCEthernetSession

IP Network

iSCSI serv

er/clie

nt

to iSCSI a

rray

iSCSI array to iSCSI array for replication

IP Network

FCIP gateway to FCIP gateway for replicationFCIP

GatewayFCIP

Gateway

Array

Array

Array

Array


STORAGE OVER IP(TCP) 솔루션

TCP/IP chattiness/statefullness

Windowing

slow start

packet loss

QoS

B/W

SACK Tuning TCP timers Compression WAN optimization

문제점 솔루션

대부분의 스토리지 기반 문제는 거리와 스토리지 특성에서 기인함


AGENDA






GbE/10GbE SERVERS

FC STORAGE

Pooled storageiSCSI / NAS

Customer B - IT DC

SRX

MX

Junos Space

DATA CENTER 모델

Public Cloud UsersSMB

GbE/10GbE SERVERSPooled Storage(NAS)

Production Data Center A

MX

GbE/10GbE SERVERS

FC STORAGE

Pooled storageiSCSI / NAS

Customer A - IT DC

SRX

MX

NATFWLB

IPSec

Junos Space

Inter Data Center Connectivity

Hybrid Cloud

Junos Space

SRX

GbE/10GbE SERVERS Pooled Storage(NAS)

Production Data Center B

MX

Hybrid Cloud VPN

NATFWLBIPSec

SRX

VPN

Junos Space

EXQFX μF

Switch Switch

EXQFX μF


DATA CENTER 들 사이의 어플리케이션 자원 가용성

고 가용 컴퓨팅 클러스터 A group of compute resources providing compute services for an application Shared data source

확장된 / 지리적 클러스터 Members of compute clusters are located across DC sites Replicated data source across DC sites

가상화된 컴퓨팅 클러스터 Virtual machine mobility among compute clusters

클러스터 인지 어플리케이션 Application resiliency integrated with compute cluster environment


고 가용 컴퓨팅 클러스터

로컬 클러스터

Private Network

Public Network

Compute Cluster

Shared Disk

Site-1

지리적 클러스터

Private Network

Public Network

Compute Cluster

Disk-1

Site-1

Disk-2

Site-2

Data Replication


가상화된 컴퓨팅 클러스터

클러스터 환경에 가상 머신 사용

클러스터 그룹내에서 가상 머신의 이동성에 의해 민첩성 향상

다수의 네트워크들이 필요

HA/DR 의 능력을 올리기 위해 DATA CENTER 들 간의 클러스터 구성

Live Migration Network

Heartbeat Network

Public Network


고 가용성 컴퓨팅 클러스터 네트워킹

결정적 레이턴시 Heartbeat communication Virtual machine memory and state replication

네트워크 처리 성능 Virtual machine memory and state replication VM image transfer

신뢰성 및 복원력 To avoid Split brain conditions

네트워크 트래픽 분리 QoS guarantee

Layer-2 연결성 For single hop heartbeat For simpler user session redirection


네트워크 가용성

네트워크 장비 고 가용성 Redundant components Graceful Routing Engine fail-over Non stop routing/bridging Modular software Routing and Forwarding on Separate Planes

네트워크 링크 고 가용성 Link Aggregation Virtual Chassis Multi Chassis LAG Bidirectional Forwarding Detection Fast Reroute


데이터 세션 리다이렉션

클러스터 멤버간 Layer-3 연결 DR site is in different IP subnet May require GSLB setup DNS update with shorter TTL Higher convergence time and hence higher RTO

클러스터 멤버간 Layer-2 연결 DR site is in same IP subnet Target IP address is same after fail-over User session redirect based on forwarding update Target machine need to learn the default gateway mac address Faster convergence – possibility of no session loss


LAYER 2 확장

VM/Workload 이동성

지리적 클러스터 / 확장된 클러스터를 위한 고 가용성

이점 : 신뢰성 , Workload 이동

문제점 : 클러스터 제어의 분리 이슈 . 양 파트의 클러스터가 통신 할 수 없을 때 , 공유 스토리지에 계속 쓸 경우 데이터의 이상이 발생 할 수 있음 .

L2 도메인의 문제가 WAN 을 통해서도 내포

대역폭의 비용

고 가용성 / 클러스터 솔루션은 LAN 을 위해 만들어 졌으므로 WAN 환경에서 잘 동작 하지 않거나 새로운 문제를 발생 시킬 수 있음 .


VMOTION 네트워크 요구 사항

Vmware 는 리던던시 및 스토리지를 위해 최소 2 개 이상의 인터페이스를 요구함 . VMKernel – Clustering interface that is used for synchronization traffic (L2 requirement by

recommendation of VMware support) Data Interface – Assumes MAC and IP address of VM upon move (L2 requirement as we would

still be assuming the same segment for IP routing)

거리 제한 Distance limitation is based upon local function and memory paging synchronization goal for

‘no interruption of service’ (5ms ~100km)

동기화를 위해 서버의 트래픽 로드가 크게 증가 할 수 있음


DATA CENTER 간의 VMOTION 이슈

각 DC 에서 같은 클러스터와 호스트를 관리 하기 위해 같은 vCenter 를 사용하여야함

멀티티어 어플리케이션 move one server need to move them all or need to still ensure communication to them all

ESXi 호스트 당 대역폭 및 QoS move one VM or move all of them. Requires a lot of bandwidth no QoS differentiation between VMs on

vswitch/LAN – Bandwidth cost tends to be prohibitively expensive long distance remember the data is the most important thing if you don’t shift this then many applications are useless

방화벽과 로드 밸런서들의 세션 정보는 DC 간 유지 및 공유가 필요 할 수 있음


AGENDA






MX 는 DC LAN 및 WAN 의 연결성 제공

WAN

LANMX supporting extensive set of LAN features

High scale, multi-tenancy, resiliency, deployment flexibility

Inline services, stateful services

WAN / CORE MX providing market leading WAN features

Proven platform

Over 24,000 chassis shipped

Over $3B revenues

Over 2,500 customers

EDGE

COLLAPSED

CORE


DCI MX – L2 와 L3 연결성 제공

L2 와 L3 DCI 로 표준 기술 사용

고 확장성의 멀티태넌시 제공

SRX5800EX4200

EX/MX

MX Series

MX SeriesRemote

Data Center

MX Series

MX Series Remote

Data Center

VPLS and L3VPN over MPLS (or) IP

GbE/10GbE SERVERS

NATFWLB

IPSec

SRX

Switch

GbE/10GbE SERVERS

멀티 벤더 솔루션

강력한 고가용성 기능과 빠른 복구 능력 제공

테스트되어진 , 적용되어진 , 입증된 솔루션


MX 를 통한 L2 DCI – VPLS

특성 : Forwarding of Ethernet Frames Forwarding of Unicast frames with an unknown MAC address Replication of broadcast and multicast frames Loop prevention Dynamic Learning of MAC address

IP of MPLS

Site 1 Site 2

MULTIVENDOR – PROVEN – FAST RECOVERING – HIGH SCALEL2 CONNECTIVITY

VPLS Edge

VPLS Edge


Virtual Private LAN Service (VPLS) 는 공유된 IP/MPLS 네트워크를 통하여 VLAN 확장을 제공함

VPLS 특성

Full Mesh

VLAN Separation

Provisioning

Multicast, Broadcast and Flooding

Availability

Any-to-Any connectivity regardless of physical path

Separate VPLS instances per VLAN. Allows network-wide segmentation with very large scale

New site Auto Discovery

Scale forwarding with Multicast & Point-to-Multipoint capabilities

Underlying MPLS offers ECMP, Fast Reroute


LAN

ETHERNET VPN 소개

LAG

Ethernet-VPN 은 새로운 표준 기반의 프로토콜

L2 도메인을 MPLS 또는 IP 백본을 통해 연결

멀티 벤더에서 지원

Leverages BGP 와 MPLS 의 강점을 접목

Policy based learning, advertisement for controlled learning

L3 aware L2 solution

WAN

BGP based state exchange

EVPN router

EVPN router


PRIVATE MPLS WAN without EVPN

VLAN 10

DATA CENTER 1

VLAN 10

DATA CENTER 2

✕

MAC VLAN Interfaces

AA 10 xe-1/0/0.10

Router 1’s MAC Table

MAC: AA

Server 1xe-1/0/0.10

xe-1/0/0.10 xe-1/0/0.10

xe-1/0/0.10

MAC: BB

Server 2ge-1/0/0.10

ge-1/0/0.10

MAC VLAN Interfaces

BB 10 xe-1/0/0.10


ge-1/0/0.10

ge-1/0/0.10

PRE-EVPN: LAYER 2 STRETCH BETWEEN DATA CENTERS Without EVPN

Data Plane

• Only one path can be active at a given time• Remaining links are put into standby mode

Control Plane

• Layer 2 MAC tables are populated via the data plane (similar to a traditional L2 switch)

• Results in flooding of packets across WAN due to out of sync MAC tables


PRIVATE MPLS WAN without EVPN

VLAN 10

DATA CENTER 1

VLAN 10

DATA CENTER 2

MAC VLAN Interfaces

AA 10 xe-1/0/0.10

BB 10 ge-1/0/0.10


MAC: AA

Server 1xe-1/0/0.10

xe-1/0/0.10 xe-1/0/0.10

xe-1/0/0.10

MAC: BB

Server 2ge-1/0/0.10

ge-1/0/0.10

MAC VLAN Interfaces

BB 10 xe-1/0/0.10

AA 10 ge-1/0/0.10


ge-1/0/0.10

ge-1/0/0.10

With EVPNData Plane

• All paths are active• Inter-data center traffic is load-balanced across all

WAN links

Control Plane

• Layer 2 MAC tables are populated via the control plane (similar to QFabric)

• Eliminates flooding by maintaining MAC table synchronization between all EVPN nodes

POST-EVPN: LAYER 2 STRETCH BETWEEN DATA CENTERS


VM MOBILITY TRAFFIC OPTIMIZER By utilizing the control plane learning that EVPN offers, Juniper is able build upon that

technology and bring a significant set of enhancements for customers that are providing Layer 2 stretch but are plagued with the effects of a scenario commonly referred to as “Trombone Routing.”

This is a result of a VLAN being “stretched” to reside in two or more data centers but with only one optimal way into the VLAN from the outside (IGP route preference) and one optimal way out (single Master VRRP address).

VMTO fixes both of these problems by:

1) Optimizing routing information sent to the WAN that is specific to the location of each VM

2) Ensuring every router within the VLAN has an active instance of the default gateway

Ultimately, this allows a customer to provide a better experience to their end users while being able to take advantage of all resources—efficiently and in multiple geographies.


Scenario with VMTO enabled

PRIVATE MPLS WAN PRIVATE MPLS WAN

VLAN 10 VLAN 10 VLAN 10VLAN 10

Scenario without VMTO

THE NEED FOR L2 LOCATION AWARENESS


DC 2 VLAN 10

10.10.10.100/24

DC 3

10.10.10.200/24

VLAN 10

VLAN 20

Server 2 Server 3

Server 1

PRIVATE MPLS WAN

DC 1

20.20.20.100/24

Active VRRPDG: 10.10.10.1

Standby VRRPDG: 10.10.10.1



Task: Server 3 in Data Center 3 needs to send packets to Server 1 in Data Center 1.Problem: Server 3’s active Default Gateway for VLAN 10 is in Data Center 2.Effect: 1. Traffic must travel via Layer 2 from Data Center 3 to

Data Center 2 to reach VLAN 10’s active Default Gateway.

2. The packet must reach the Default Gateway in order to be routed towards Data Center 1. This results in duplicate traffic on WAN links and suboptimal routing – hence the “Egress Trombone Effect.”

WITHOUT VMTO: EGRESS TROMBONE EFFECT


DC 2 VLAN 10

10.10.10.100/24

DC 3

10.10.10.200/24

VLAN 10

VLAN 20

Server 2 Server 3

Server 1

PRIVATE MPLS WAN

DC 1

20.20.20.100/24

Active RVIDG: 10.10.10.1




Task: Server 3 in Datacenter 3 needs to send packets to Server 1 in Datacenter 1.

Solution: Virtualize and distribute the Default Gateway so it is active on every router that participates in the VLAN.Effect: 1. Egress packets can be sent to any router on VLAN 10

allowing the routing to be done in the local datacenter. This eliminates the “Egress Trombone Effect” and creates the most optimal forwarding path for the Inter-DC traffic.

WITH VMTO: NO EGRESS TROMBONE EFFECT


DC 2 VLAN 10

10.10.10.100/24

DC 3

10.10.10.200/24

VLAN 10

VLAN 20

Server 2 Server 3

Server 1

PRIVATE MPLS WAN

DC 1

20.20.20.100/24

Task: Server 1 in Datacenter 1 needs to send packets to Server 3 in Datacenter 3.Problem: Datacenter 1’s edge router prefers the path to Datacenter 2 for the 10.10.10.0/24 subnet. It has no knowledge of individual host IPs.

Effect:1. Traffic from Server 1 is first routed across the WAN to

Datacenter 2 due to a lower cost route for the 10.10.10.0/24 subnet.

2. Then the edge router in Datacenter 2 will send the packet via Layer 2 to Datacenter 3.

10.10.10.0/24 Cost 5 10.10.10.0/24 Cost 10

Route Mask Cost Next Hop

10.10.10.0 24 5 Datacenter 2

10.10.10.0 24 10 Datacenter 3

DC 1’s Edge Router Table Without VMTO

WITHOUT VMTO: INGRESS TROMBONE EFFECT


DC 2 VLAN 10

10.10.10.100/24

DC 3

10.10.10.200/24

VLAN 10

VLAN 20

Server 2 Server 3

Server 1

PRIVATE MPLS WAN

DC 1

20.20.20.100/24

Effect: 1. Ingress traffic destined for Server 3 is sent directly across

the WAN from Datacenter 1 to Datacenter 3. This eliminates the “Ingress Trombone Effect” and creates the most optimal forwarding path for the Inter-DC traffic.

Task: Server 1 in Datacenter 1 needs to send packets to Server 3 in Datacenter 3.

Solution: In addition to sending a summary route of 10.10.10.0/24 the datacenter edge routers also send host routes which represent the location of local servers.

10.10.10.0/24 Cost 5 10.10.10.0/24 Cost 10

Route Mask Cost Next Hop

10.10.10.0 24 5 Datacenter 2

10.10.10.0 24 10 Datacenter 3

10.10.10.100 32 5 Datacenter 2

10.10.10.200 32 5 Datacenter 3

DC 1’s Edge Router Table WITH VMTO

10.10.10.100/32 Cost 5 10.10.10.200/32 Cost 5

WITH VMTO: NO INGRESS TROMBONE EFFECT


JUNIPER SOLUTIONS FOR VM MOBILITY

L2 & L3 address no longer pinned to a site, interface

Ingress and Egress traffic convergence, optimization

Learning and information distribution control

L2 & L3 interaction for best user experience

Fast convergence of network paths as VM moves

Challenges

L2: Split subnet supported by VPLS

L3: Need provisioning help to advertise split subnet members

L2 and L3: Split subnet supported by EVPN and L3VPN

VPLS & L3VPN EVPN & L3VPN

L2: MX implements integrated L2 and ARP learning (DP)

Fast convergence through flooding

Ingress, egress L2: automatic

Ingress L3: provisioning based

Egress L3: VRRP leverage

L2: DP based learning no advertisement

L3: BGP policies

Limited

L2: MX implements integrated L2, ARP, L3 advertisement (DP, CP)

Convergence through flooding and CP announcement

Ingress, Egress, L3, L3 automatic

L2 and L3: BGP policies

Full


WAN

VXLAN ON MX – OVERLAY WITHIN LAN

VC

VC VC

POD-1: Hypervisor based environment, terminating VXLAN

tunnels on MX and on virtual-switch of servers

POD-2: Legacy server environment, terminating VXLAN tunnels on MX

and access switches

POD-3: Legacy server and LAN environment with no VXLAN tunnels,

MX providing gateway function to VXLAN environment

Virtualized L2, and L3

Bridge-domain, virtual-switch, IRB, L2, and L3

Bridge-domain, virtual-switch, IRB, L2, and L3 Bridge-domain,

virtual-switch, IRB, L2, and L3

MX acting as the VTEP for VXLAN,

legacy LAN with full BD, VS, IRB, L2, L3

support

High scale multitenant VXLAN implementation

Orchestration & Controller

Virtualized L2, and L3

DC GW

Intra DC Network

Intra DC Network

TOR TOR


THANK YOU

Documents

Dr관련 세미나 자료 v2333