6th SDN Interest Group Seminar - Session1 (131210)

1 © 2013 Arista Networks. All rights reserved.

김창민 부장 시스템 엔지니어[email protected]


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

Change through Innovation

Arista


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

§  Founded in 2005 – FCS 2008 §  Focused on Data Center L2/L3 Ethernet switching solutions §  Pre-IPO, Privately held and profitable

>2000 customers globally §  Successful franchise in many of the world’s largest DCs and highest

traffic sites §  Award Winning Solutions

Arista - Who are we?


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

Data Centers ARE

Demanding

Complex Growing

CHANGING


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

Open Source

Cloud Computing

DevOps Tools

Automation

Software Defined Networking

Network Virtualization

Commodity Hardware

Network Programmability

Scale-out Design

Server Virtualization

Keywords for Today’s Datacenter Market

OpenFlow


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

For the next decade Ethernet will be the primary data center/server interconnect


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

Source: Dell’Oro, Jan’13 5 Year Forecast

Ethernet for Datacenter Market

•  Faster CPUs need Faster Networks •  Intel Sandybridge driving 10GbE adoption •  50% attach rate 2013, 80% by 2015

•  10/40/100G Market growing rapidly

•  $4B in 2010 to $16B in 2016 •  From 5M ports 2010 to 67M ports 2016

•  Faster End nodes need faster Backbones

•  Many apps drive east/west traffic not north/south •  Cluster sizes getting larger & larger


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

Drivers for Increasing Network Bandwidth


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

Moore’s Law and Networking

Semiconductor Technology Roadmap

Moore’s Law 1971-2011

64-bit CPU Cores over Time


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

1990+ Proprietary ASICs 2000+ FPGA Processing 2010+ Full Custom Silicon

-  Expensive to produce

-  Relatively slow (low transistor res)

-  Low density

-  Power hungry

-  Long write-down

Complex to program

Low port density/chip

Result in high component count, �� low feature velocity

Ultimate flexibility

Standardized features

High clock rates and density

Competitive market

Exceptional innovation rate

Platforms Optimized for Custom Silicon


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

Evolution of Custom Switch Silicon

Technology 130nm 65nm 40nm 28nm 10G ports 24 64 128 256

Throughput 360M PPS 960M PPS 2B PPS 4B PPS

Buffer Size 2 MB 8 MB 16 MB 32 MB

Table Size 16K 64K 128K 256K

Port Speeds 10G 10G/40G 10G/40G/100G 10G/40G/100G

Availability 2007 2011 2013 2015

Improvement - 3X/4Y 2X/2Y 2X/2Y

Next generation custom switch silicon is on Moore’s Law


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

Broadcom Delivering Highly integrated Silicon


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

Next generation custom switch silicon is on Moore’s Law

Moore’s Law and Networking


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

Extensible Operating System

7048T

48-port Data Center Class

Gigabit Ethernet Switch

7150S & 7124FX

Ultra Low Latency 24,52,64-port SFP

+ 1G-40GbE Switches

Intelligent

Application Switch

7050 S/T/Q

1/10G & 10/40G Data Center

Switches 10G SFP+ / 10G-T

Dense Virtualization

10GbE / 40GbE DC

7050X & 7250X

Dense Low Latency

32 & 64-port QSFP+

96xSFP+/8xQSFP+

Advanced Virtualization

Scale-out Visibility

7500E

Lossless, High Density, Modular Switching System supporting up to 1152 Wirespeed

10GbE Ports

Spine 10-40-100GbE

7300X

High Density, Modular System supporting up to

512 40GbE

Cloud Scale Leaf and Spine 10GbE-40GbE

NEW

Arista Networks: The Best Data Center Portfolio


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82 S

oftw

are

Spi

ne/L

eaf H

ardw

are

Layer 2 / Layer 3

2010 2011 2012 2013+

Modular Switch Scale

Fixed Switch Scale

7050S 64x10G

7148SX 48x10G

7050SX 128x10G

384 10GbE 2048 10G 512 40G

EOS MLAG ECMP 16-Way SDN

64-Way Ongoing Scale

7150S 64x10G

7250QX 256x10G

1152 10G 288 40G 96 100G

Arista Scalable Platforms


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

100 to 2,000 100 to 10,000 100 to 100,000+ 100 to 100,000+ Server Scale:

Layer 2 / MLAG

Layer 3 / ECMP

Spline™ L2 over Layer 3 VXLAN

Servers Servers Middle of Row Servers Servers Servers

Cloud Networking

2-tier Spine/Leaf and introducing Single-tier Spline (Collapsed Spine/Leaf)


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82 Flexible forwarding requires flexible ways of exposing

the underlying functionality

SDN

Besides larger tables, what else can 2X/2Y transistors be used for?


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

+

+ OR

1990’s

Evolution of Server Configuration

Today


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

Evolution of Network Configuration


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

Networks are an order of magnitude larger; automation has lagged

“Human middleware doesn’t scale!”�

Scale has changed everything…


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

Purist View a strict separation of control plane and data plane SDN = Openflow

Pragmatic View a network architecture designed to be programmed by high-level languages and APIs SDN = Programmability

A Common View SDN = Network Virtualization

What is SDN?


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

+ I2RS, ForCES, SDNP, PCE, NFV etc

OpenFlow

CLI

NetConf/YANG

XML-RPC

RESTful APIs, XMPP

vSphere APIs

OpenStack

SNMP

NetFlow, sFlow, jFlow, IPFIX, NetStream

Configuration Signaling Status/Alerts Statistics Sampling

SDN is More than Just OpenFlow


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

SDN is a software-to-infrastructure interface that allows applications to drive

infrastructure actions, delivering capabilities like

Network Virtualization, Orchestration, and Traffic

Engineering

Arista’s SDN View


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

EOS - Extensible Network O/S

Stock 2.6.31 x64 Linux Kernel

vCen

ter A

PI

KV

M -

Virtu

al M

achi

ne

sysDB - Central State Database

XM

PP

Clie

nt

LED

AS

IC D

river

s

Spa

nnin

g Tr

ee

CLI

Inte

rface

Man

ager

Rou

ting

Pro

toco

ls

Local Daemons/Extensions - C++, Python, etc

Local Scripts - Python, TCL, Shell

EOS eAPI - JSON Web Services API

Openflow 1.0/1.3 - multi-vendor services

Packaged Extensions - CloudVision, etc

Best-of-Breed API Integration - F5, Palo Alto, Splunk, etc

Direct Flow Programming - customized flow pathing

Arista’s EOS – The World’s most Extensible (programmable network) OS


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

OpenFlow support with all major controller vendors

OpenStack support with contributions to Quantum/Neutron to enable seamless provisioning

Native VMware integration into vSphere, vCloud and NSX Controller VXLAN integration

Native API integration with best-of-breed partners Enables network automation through event manager

Open to all Controllers and Programming Models


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

§  OpenFlow support has been added to Arista EOS

§  Arista has tested Openflow support on the 7050 Series with multiple controller vendors and projects- Big Switch, NEC, Floodlight, etc.

§  Arista is one of the most controller-friendly switching platform – with tested support for multiple controllers open and commercial

Controller

Openflow Protocol

OpenFlow Support


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

§  Enables direct CLI and eAPI control over specific flow switching operations

§  Extends the capabilities of OpenFlow with controller-less operation and enables per-flow pattern-matching with full control

§  Enables firewall load balancing, purpose-built backup network consolidation, etc.

Arista eAPI or CLI

Arista DirectFlow Control


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

switch# configure terminal!switch(config)# openflow!switch(config-openflow)# flow foo!switch(config-openflow-foo)#match ?! cos Specify the class of service! destination Destination! ethertype Specify the EtherType! icmp Internet Control Message Protocol! input Source interface! ip Internet Protocol! source Source! vlan Specify a valid VLAN! <cr> !!Switch(config-openflow-foo)#action ?! drop Drop flow! egress Mirror flows at egress! ingress Mirror flows at ingress! output destination for flows! set Set a field in the packet! transmit Specify the transmit direction queue!



R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

#!/usr/bin/perl!use JSON::RPC::Client;!!my $client = new JSON::RPC::Client;!my $request = $client->call(“https://user:passw0rd@switch/command-api”, { jsonrpc => "2.0", method => "runCmds", id => 1, params => { version => 1,! cmds => [ "configure terminal",! “openflow“, “flow foo“,! “match ip source address 192.168.1.0/24“,! “action set output interface ethernet2,ethernet3" ], format => “json" }});!!warn $client->status_line if (!$request); warn $request->error_message if ($request->is_error);!



R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

§  Arista EOS connects to Openstack - This enables automated provisioning of network segments

§  As new customers and workloads are added the network automatically provisions VLANs or VXLANs and assigns the appropriate addresses

§  Customer turn-up takes seconds rather than weeks EOS - Extensible Network O/S

Stock 2.6.31 x64 Linux Kernel

vCen

ter A

PI

KV

M -

Virtu

al M

achi

ne

sysDB - Central State Database

XM

PP

Clie

nt

LED

AS

IC D

river

s

Spa

nnin

g Tr

ee

CLI

Inte

rface

Man

ager

Rou

ting

Pro

toco

ls

Openstack Integration


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

§  Openstack Quantum only supported a single network device- commonly the Open vSwitch plugin

§  Arista developed and contributed a multi-vendor hardware driver architecture in Quantum/Neutron that enables concurrent physical and virtual network element provisioning

§  This makes Openstack Quantum/Neutron deployable and integrated across real world networks

OVS Virtual Driver

Physical Driver

Quantum/Neutron

Nova

Openstack Development and Contribution


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

from jsonrpclib import Server, ProtocolError switch = Server( "https://username:passw0rd@myswitch/eapi/v1" ) try: result = switch.runCli( cmds=[ "configure", “interface ethernet2”,

"switchport access vlan 3", "show vlan 3" ] ) except ProtocolError as e: print "Error: %s" % e.message vlan3Interfaces = result[ -1 ][ "3" ][ "interfaces" ] print "Vlan 3 now has the following interfaces %r" % vlan3Interfaces

EAPI is easy to use….

bash$ ssh username@myswitch Password: <passw0rd> myswitch> enable myswitch# configure terminal myswitch(config)# management eapi myswitch(config-mgmt-eapi)# no https shutdown

…. and easy to configure!

§  Built around the EOS CLI §  EAPI uses JSON-RPC to communicate

between application and EOS §  JSON-RPC provides an industry

standard, lightweight protocol to handle communications

§  Messaging is achieved over HTTP / HTTPS as the transport

§  Commands are sent using EOS standard CLI syntax:

‘interface ethernet2’ ‘switchport access vlan 3’

§  All output is returned to the application in standard JSON encoding messages

EOS Application Programing Interface


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

Major Online Cloud Provider

§  Company used SDN to simplify their operational process §  SDN networking is at the heart of their online services §  Significantly improved the size/scope/flexibility while reducing capex & opex

Arista EOS : ZTP, CloudVision, BGP, L3 ECMP

ROI

Practical SDN Example - Use Case #1


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

Network Tap Aggregation/Steering

Arista EOS: DANZ or Openflow-Enabled Tap Aggregation 7X50 switches

IS-IS BGP

OSPF MLAG

PIM-SM

eAPI or

CLI

Controller Mode

Direct Flow or DANZ Mode

Control Plane

Topology Plane



R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

§  By programing the ‘hash mask’ flows in an symmetric manner (source to destination) and (destination to source) we are able to maintain symmetry while scaling horizontally

§  A simple script triggered when an interface comes on line programs the flows into hardware, on interface down the script removes the flows

§  Using priority values interface failure simply shifts the flows to the next device


Scale with Symmetry


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

10.11.11.0/24 via Spine-A

10.11.11.0/24 via Leaf-B

Spine-A

10.11.11.2 10.10.10.2

Spine-B Spine-C

Backup HTTP SMTP/Mail SIP/Voice

Leaf-B Leaf-A


Time directed flows


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

10.11.11.0/24 via Spine-A @1800-2400 Backup via Spine-B

10.11.11.0/24 via Spine-C @1800-2400 Backup via Spine-C Spine-A

10.11.11.2 10.10.10.2

Spine-B Spine-C

Backup HTTP SMTP/Mail SIP/Voice

Leaf-B Leaf-A


Time directed flows


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

VXLAN – Layer 2 over Layer 3

...

...

Controller

•  All Active Multipath using ECMP (up to 32-way) •  Single L3 network for all applications

•  L2 extensions for stateful VM – VM over Layer 3 •  Extends L2 with exceptional scalability (16M Virtual NWs) •  Cloudburst over Public infrastructure


VXLAN L2 Extension


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

Software VTEP VTEP within the

hypervisor for VM to VM

conversations

Hardware VTEP Provide support for Bare Metal Servers, Storage and performance VM

Optional Component Open support for third –party

Network Virtualization Controllers

Leaf Spine Scalable DC ‘IP

Fabric’

§  Leaf-Spine IP Fabric -  Scalable Layer 3 ECMP fabric for horizontal traffic growth of Server to Server communication

§  Software VTEP -  VXLAN VTEP component within virtual switch of the hypervisor for VM to VM communication

§  Hardware VTEP -  VTEP component within the Leaf/Spine switches for bare metal servers, network services, performance requirements

or lack VTEP support in hypervisor

§  Network Virtualization Controllers -  Open API(JSON, OVSDB, Neutron) for open integration into third party NVC controllers for simplified end-to-end orchestration


VXLAN L2 Extension


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

“OpenFlow has helped us improve backbone performance and reduce backbone complexity and cost.” Urs Hoelzle, Google (Google Fellow, SVP Technical Infrastructure)

2010 Cisco CRS3 router 32 x 100G 3 racks (132RU), >25KW power [capex estimate $5M]

2013 Arista 7500 switch 96 x 100G 11RU, <4KW power [capex ~$300K @ 36x100G, $641K @ 96x100G]

2012 Juniper PTX P router 32 x 100G 35RU, 10.4KW power [capex estimate $2.5M]

<1 year: 3x more ports in 3x less space and 2.5x less power (absolute terms: 9x density & 7.5x power)

2 years: 3x less space 2.5x less power same density

http://www.opennetsummit.org/archives/apr12/hoelzle-tue-openflow.pdf


Router Ports vs Switch Port Economics


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82 Arista EOS

Programmable Foundation

Self Healing Event Driven

Open

Phase 1 – 2010+

Open APIs – Control & Data Orchestration Integration

Rapid Service Delivery

Phase 2 – 2012+

Ubiquitous Network-wide Virtualization All devices virtual and

SW provisioned Phase 3 – 2014+

Arista SDN Phases: Making SDN a Reality


R:59 G:78 B:111

R:22 G:33 B:51

R:171 G:163 B:188

R:83 G:110 B:127

R:29 G:37 B:82

Those deploying SDN successfully will be doing so to solve real business problems that couldn’t otherwise be solved in a time or cost effective manner using existing tools or paradigms

Summary