251 Brad Hedlund

8/18/2019 251 Brad Hedlund

1/26

Global Marketing

Architecting Data CenterNetworks in the era of Big Dataand Cloud

Brad Hedlund

Spring Interop—

May 2012


2/26

Global Marketing

• Centralized, Scale-up Layer 2 networks• Monstrous chassis switches

2

Distributed, Scale-out Layer 3 fabricsEfficient fixed switchesOpen, industry standard protocols

TRILL OpenFlow VEPA SPB

THE SAME OLD

Or a Different Approach

Two approaches to DC Networking

Brad Hedlund


3/26

Global Marketing

Networks that suck for Cloud & Big Data

3

PARTITIONED CAPACITY

Core

Dist

Access

“Data center networks are in my way” -James Hamilton, AWS

VM

NetworkTopology

CapacityTopology

Brad Hedlund


4/26

Global Marketing

Networks that Don’t suck for Cloud & Big Data

4

UNIFORM CAPACITY

Spine

Leaf

All points equidistant

VM

NetworkTopology

CapacityTopology

Brad Hedlund


5/26

Global Marketing

Rack 2

NodeNode

Node

Node

Job Tracker

Rack 1

NodeNode

Node

Node

Name Node

Rack N

Node

Node

Node

Node

Node

switch

Big Data

• Inverse Virtualization• Workloads orchestrated like cattle• L2 or L3 network. Does it matter?

5

Rack 3

NodeNode

Node

Node

Secondary NN

Rack 4

World

Node

NodeNode

Node

Client

switch switch

switch

switch switch

switch

TCP

TCP

TCPClient

TCP

Brad Hedlund


6/26

Global Marketing

Basic requirements of Cloud (IaaS)

• Secure, Scalable Multi Tenancy

• Location independence

• On Demand virtual networks

6

VM VM

FW

VM VM

LB

switch switch

switch switch

switch switch

Physical

Network

VirtualNetwork

World

Brad Hedlund


7/26

Global Marketing

Blend the Virtual and Physical Networks

•Tenant subnet = Network VLAN

7

VM VMVM VM

switch

switch switch

VM VM

VLAN 10

VLAN 20

Host Host

vSwitchvSwitch

VM VM

Brad Hedlund


8/26

Global Marketing

Abstract the Virtual Network from Physical

•Network Virtualization Overlay•Tenant subnet = Software VLAN

8

VM VMVM VM

switch

switch switch

VM VM

Host Host

vSwitchvSwitch

VM VM

Segment ID 20

Segment ID 10

Brad Hedlund


9/26

Global Marketing

Scale-up centralized Layer 2

• 2-post Rooted Architecture

• Centralized L2/L3

• L2/L3/ARP table scale?

• Scale w/ Bigger Boxes

• Precious Pets

• VLAN Provisioning?

• Broadcasts

9

VM VMVMVM

vSwitchvSwitch

L3

L2

Brad Hedlund


10/26

Global Marketing

(16)

(2)(8)

(64)

1980 Server ports

Scale-out Layer 3 Leaf/Spine Fabric

• Mesh from Leaf to Spine

• OSPF, ISIS, BGP, TRILL

• ToR w/ 16 uplinks (ECMP)

10

768 Server ports3072 Server ports6144 Server ports

(16)

(128)

• Non-blocking Spine

• 3:1 @ ToR

• 128 port 2RU Spine

L3

L2

Brad Hedlund


11/26

Global MarketingBrad Hedlund

6144 Server ports

(16)

(2)(8)

(64)

Uniform fabric for Cloud & Big Data

11

L3

L2

(16)

(128)

VM VMVMVM

vSwitchvSwitch

Rack 3Rack 1

Name Node

Rack 2

Job Tracker

Rack N

Secondary NNNode

Node

Node

Node

Node

Client

Node

Node

Node

Node

Client

Node

Node

Node

Node

Node

Node

Node

Node

Node

Node

Node

Node

Node

Node

Block I/ONASObject

Storage AccessHadoop

Database


12/26


(16)

(2)(8)

(64)

Attaching Services & North/South

12

(16)

(128)

FirewallFirewall

World

LBLB

vswitchVM VM VM

vswitchVM VM VM

vswitchVM VM VM

vswitchVM VM VM

vswitchVM VM VM

vswitchVM VM VMRack 1 Rack N

Node

Node

Node

Node

Node

Client

Node

Node

Node

Node

Client

Name Node Job Tracker

Secondary NN

L3

L2

x86 Gateways


13/26

Global Marketing

Generic Logical Architecture 1

13

World

Brad Hedlund

FW

LB

FW

LB

VM VM VM

Green Co. Orange Co.

L3

NATL3

NAT

L2

L3

L2

L2

Fabric DC router• Overlay based L2•

Physical/Static FW

VM VM

Big Data

L2


14/26

Global Marketing


14

World

Brad Hedlund

FW

LB

FW

LB

VM VM VM


L3

NATL3

NAT

L2

L3

L2

L2

Fabric DC router• Overlay based L2• Virtual/Mobile FW• Overlay Gateway

Pub DMZ

Big Data

VM VM

L2


15/26

Global Marketing

Density: Fixed vs. Chassis

0

20

40

60

80

100

120

140

2008 2010 2012 2014

Chassis

Fixed

15

10G per RU @ Line Rate (L3)

Brad Hedlund


16/26

Global Marketing

Power: Fixed vs. Chassis

0

2

4

6

8

10

12

14

16

18

2010 2012 2014

Chassis

Fixed

16

Max Watts / Line Rate 10G (L3)

Brad Hedlund


17/26


(16)

(2)(8)

What are the Challenges?

(16)

(128)

• Deployment & Cabling

• Configuration & Policy

• Monitor & Troubleshoot

• Layer 2 (TRILL?)

• Design Best Practices

Dell Fabric Manager

L3L2

17

BOOTH DEMODell Fabric Manager

Validate deployment

Automate fabric configuration Monitoring & Operations


18/26

Global Marketing

Webinar: CLOS Fabrics Explained

18 Brad Hedlund

http://closfabric.eventbrite.com/ Wednesday, June 20, 2012 from 10:00 AM to 1:00 PM (ET)

HOST

CO-HOST

Ivan Peplenjak

Yours Truly

DISCOUNT CODE: BradInterop

http://closfabric.eventbrite.com/http://closfabric.eventbrite.com/


19/26

The power to do more

Visit us at booth #1727

19


20/26

Global Marketing


20

World

Brad Hedlund

FW

LB

FW

LB

VM VM VM


L3

NATL3

NAT

L2

L3

L2

L2

Fabric DC router• No Overlays• TRILL based L2• Virtual/Mobile FW

Pub DMZ

TRILL

Big Data

VM VM

L2


21/26

Global Marketing

(2)(8)

Three Stage Layer 3 Leaf/Spine Fabric

21

(64)

(128)

L3

L2

24,576 Server ports

(512)

• Non-blocking @ top tiers

• Default route @ ToR & Leaf

• Leaf+ToR mesh groups

• ~8usec worst case

0/0

0/0

/26 /26

/26

Brad Hedlund


22/26

Global Marketing

(2)(8)

8192 non-blocking Fabric

22

(64)

(128) L3

L2

8192 non-blocking

Brad Hedlund


23/26

Global Marketing

(2)(32)


23

(64)L3

L2

4096 non-blocking

Brad Hedlund


24/26

Global Marketing

(2)(16)


24

(32)L3

L2

2048 non-blocking

Brad Hedlund


25/26

Global Marketing

(2)

(4)


25

256 non-blocking

Brad Hedlund

Th f 40G QSFP i h


26/26

Global Marketing

The case for 40G QSFP switch ports

26

QSFP

SFP+ SFP+ SFP+ SFP+

10G10G10G10G 10G 10G 10G 10G

VS

$1,800$1K $1K $1K $1K

Brad Hedlund

32 ToR

$512K$230K

Documents

251 Brad Hedlund