Introduction to OpenDaylight: Current Events and OpenStack Neutron ...dmm/talks/2015/openstack_austin.pdf · Introduction to OpenDaylight: Current Events and OpenStack Neutron Integration

Introduction to OpenDaylight: Current Events and OpenStack Neutron Integration David Meyer Chief Scientist and SP CTO @Brocade Research Scientist @University of Oregon Chair of the OpenDaylight Board of Directors dmm@{brocade.com, uoregon.edu, 1-4-5.net, ….} Note: This deck contains slides courtesy Colin Dixon and a cast of 1000s

Agenda • What is OpenDaylight? • History: Helium • What’s in store for Lithium • Some Personal Learnings • Neutron Integra>on? • Q&A

What is OpenDaylight OpenDaylight is an Open Source So+ware project under the Linux Founda3on with the goal of furthering the adop>on and innova>on of So+ware Defined Networking (SDN) through the crea>on of a common industry supported plaGorm

Code Acceptance Community

To create a robust, extensible, open source code base that covers the major common components required to build an SDN solu>on

To get broad industry acceptance amongst vendors and users

• Using OpenDaylight code directly or through vendor products

• Vendors using OpenDaylight code as part of commercial products

To have a thriving and growing technical community contribu>ng to the code base, using the code in commercial products, and adding value above, below and around.

Who is OpenDaylight?

Who is OpenDaylight? (Really)

•  Like any Open Source Project, OpenDaylight primarily consists of those who show up to do the work.

• Running around 150–200 commits per week •  30 Days: ~400 commits, ~55 contributors

•  During releases this is >= 1000 commits and >= 100 commi\ers •  12 Months: ~10,000 commits, ~260 contributors

•  Strong integra>on and tes>ng community •  This stuff really ma\ers •  Staffing I&T is well, challenging

5

Source: https://www.openhub.net/p/opendaylight

Why Open Source?

•  Short version: this is how modern infrastructure is built •  Modern way to build “undifferen>ated” plumbing

•  Long version: •  Build more, be\er code faster via collabora>on •  Make be\er decisions with devs and users at the table •  Spend more >me on the code that ma\ers

•  80/20 rule: 80% of code is non-‐differen>a>ng 6

Aside: Release Naming

7

Brief Note on Project Lifecycles

8


AAA: Authen>ca>on, Authoriza>on & Accoun>ng AuthN: Authen>ca>on BGP: Border Gateway Protocol COPS: Common Open Policy Service DLUX: OpenDaylight User Experience DDoS: Distributed Denial Of Service DOCSIS: Data Over Cable Service Interface Specifica>on FRM: Forwarding Rules Manager GBP: Group Based Policy LISP: Locator/Iden>fier Separa>on Protocol

OVSDB: Open vSwitch DataBase Protocol PCEP: Path Computa>on Element Communica>on Protocol PCMM: Packet Cable Mul>Media Plugin2OC: Plugin To OpenContrail SDNI: SDN Interface (Cross-‐Controller Federa>on) SFC: Service Func>on Chaining SNBI: Secure Network Bootstrapping Infrastructure SNMP: Simple Network Management Protocol TTP: Table Type Pa\erns VTN: Virtual Tenant Network

Legend “HELIUM”

AAA AuthN Filter

MD-‐SAL RESTCONF (REST) APIs

OpenFlow Enabled Devices

DLUX VTN Coordinator

OpenStack (via Neutron)

SDNI Wrapper

DDoS Protec3on

Network Applica3ons Orchestra3ons & Services

Open vSwitches Addi3onal Virtual & Physical Devices

Data Plane Elements (Virtual Switches, Physical

Device Interfaces)

Controller PlaZorm Topology Manager

Stats Manager

Switch Manager

Fwding Rules Mgr

Host Tracker

API-‐Driven Service Abstrac3on Layer (AD-‐SAL)

GBP Service

Service Flow

Chaining

DOCSIS Service

VTN Plugin20C

LISP Service

L2 Switch

SDNI Aggregato

r

Base Network Service Func3ons

OVSDB

OVSDB NETCONF PCMM/COPS SNBI LISP BGP PCEP SNMP Plugin20C Southbound Interfaces &

Protocol Plugins

Custom Basic AuthN Filter

AD-‐SAL REST APIs

Neutron AuthN

Neutron APIs

OpenStack Neutron Service

Model-‐Driven Service Abstrac3on Layer (MD-‐SAL) Service Abstrac3on Layer (Common models, APIs,

etc.)

OpenFlow 1.0

OpenFlow 1.0 1.3 TTP

Clustering

•  Java chosen as an enterprise-‐grade, cross-‐plaGorm compa>ble language

•  Java Interfaces are used for event listening, specifica>ons and forming pa\erns

•  Maven – build system for Java

•  OSGi: •  Allows dynamically loading bundles •  Allows registering dependencies and services exported

•  For exchanging informa>on across bundles

•  Karaf: Light-‐weight Run>me for loading modules/bundles •  OSGi based. Primary distribu>on mechanism for Helium

11

OSGi Framework (Equinox)

FeatureA SAL

FeatureB …

Karaf

ODL Helium: Karaf

$  wget http://nexus.opendaylight.org/content/groups/public/org/opendaylight/integration/distribution-karaf/0.2.0-Helium/distribution-karaf-0.2.0-Helium.zip

$  unzip distribution-karaf-0.2.0-Helium.zip $  cd distribution-karaf-0.2.0-Helium $  ./bin/karaf opendaylight-user@root> feature:list (get all apps available)

opendaylight-user@root> feature:install odl-dlux-core

opendaylight-user@root> feature:install odl-openflowplugin-all

opendaylight-user@root> feature:install odl-l2switch-all

opendaylight-user@root> bundle:list | grep Active

Now your controller is ready to connect to switches and handle incoming flows.

ODL Helium: Karaf

ODL Helium: Clustering

•  The MD-‐SAL data store, no>fica>ons and RPCs now work in a cluster •  Built using the RAFT consensus algorithm on top of Akka messaging •  Tolerates f controller failures if you have 2f+1 controllers •  Uses sharding for scale-‐out performance

•  Lithium work items •  Finer-‐grained, configurable sharding •  Migra>ng plugins to take advantage of clustering and support failover •  Provide clearer models for building clustered applica>ons

• Based on modern frameworks: node.js, AngularJS

• Completely decoupled from the core controller

•  Run it from any loca>on •  Modular, easy to extend

ODL Helium: DLUX

ODL Helium: Policy

• Policy is everywhere at them moment •  Group-‐based Policy, Congress, Intent, ACI, …

• At least three policy-‐oriented projects in ODL •  Service Func>on Chaining •  Group-‐based Policy •  Network Intent Composi>on

• ODL is ac>ng as a proving ground for policy approaches where engineers and users can play with different approaches

• OpenDaylight exposes a single common OpenStack Service Northbound

•  Matches Neutron API precisely •  Mul)ple implementa)ons of Neutron in OpenDaylight

• New features in Helium •  Distributed L3 forwarding •  OpenStack Security Groups •  LBaaS implementa>on OpenDaylight

OpenContrail Provider VTN Provider

Neutron Service

OpenStack Neutron

OVSDB Provider

Neutron ML2 MechanismDriver

OpenDaylight APIs (REST)

ODL Helium: OpenStack Integration

Growth from Hydrogen to Helium

23

13 154

291

10,411 total

1.9M lines of code since projects launch

Adoption


AAA: Authen>ca>on, Authoriza>on & Accoun>ng ALTO: Applica>on Layer Traffic Op>miza>on AuthN: Authen>ca>on BGP: Border Gateway Protocol CAPWAP: Control and Provisioning of Wireless Access Points COPS: Common Open Policy Service DIDM: Device Iden>fica>on and Driver management DLUX: OpenDaylight User Experience DDoS: Distributed Denial Of Service

DOCSIS: Data Over Cable Service Interface Specifica>on FRM: Forwarding Rules Manager GBP: Group Based Policy IoTDM: Internet of Things Data Broker LACP: Link Aggrega>on Control Protocol LISP: Locator/Iden>fier Separa>on Protocol MAPLE: Maple Programming NIC: Network Intent Proposal OVSDB: Open vSwitch DataBase Protocol OPFLEX: Extensible Policy Protocol

Legend

“LITHIUM” AAA-‐ AuthN Filter

OpenDaylight APIs (REST)

OpenFlow Enabled Devices

DLUX VTN Coordinator OpenStack Neutron SDNI Wrapper DDoS Protec3on

Open vSwitches Addi3onal Virtual & Physical Devices

Topology Processing

DIDM

MD-‐SAL / Yangtools

GBP Service SFC

DOCSIS Abstrac3on

VTN Manager

Plugin20C LISP Service

BGP PCEP OVSDB

OVSDB NETCONF PCMM/COPS SNBI LISP BGP

PCEP SNMP Plugin20C OpenFlow

Neutron Service

SDNI Aggregator

Persistence

L2 Switch

TCP-‐MD5

SXP USC

Discovery IoTDM

IoT

LACP

MAPLE

ALTO

CAPWAP

Reserva3on

TSDR

VPN Service NIC

USC Manager

OPFLEX

Topology Inventory FRM

Network Applica3ons Orchestra3ons and Services

NB APIs

Applica3ons

Plugin Services

Controller plaZorm

SB interfaces & protocols plugins

PCEP: Path Computa>on Element Protocol PCMM: Packet Cable Mul>Media Plugin2OC: Plugin To OpenContrail SDNI: SDN Interface (Cross-‐Controller Federa>on) SFC: Service Func>on Chaining SNBI: Secure Network Bootstrapping Infrastructure SNMP: Simple Network Management Protocol SXP: Source-‐Group Tag eXchange Protocol TSDR: Time Series Data Repository TTP: Table Type Pa\erns USC: Unified Secure Channel VTN: Virtual Tenant Network

Lithium Dependency Graph


Key Personal Learning: Open Source is the Modern Way to Develop Non-Differentiated “Plumbing”

•  Community building is a core Open Source objective •  Both intra and inter project

•  Code is the coin of the realm •  But don’t forget the importance of testing and integration, documentation, …

Putting this all together à 23

•  Engineering systems are as important as artifacts

Implica6on: Engineering ar6facts are no longer the source of sustainable advantage and/or innova6on

•  Engineering Systems •  Culture •  People/Process

Perhaps surprisingly, the “hyper-‐scale” and open source communi>es have taught me that actual ar>facts (in our case network applica>ons as well as HW/SW) are ephemeral en>>es and that the only source of sustainable advantage/innova>on consists of

24

http://www.sdncentral.com/education/david-meyer-reflections-opendaylight-open-source-project-brocade/2014/03/

Bio-‐techno Convergence and The Hidden Nature of Complexity

David Meyer CTO and Chief Scien>st, Brocade

Director, Advanced Technology Center, University of Oregon Network Complexity Research Group

IETF 88 Vancouver, BC

dmm@{brocade.com,uoregon.edu,1-‐4-‐5.net,…}

hcp://www.1-‐4-‐5.net/~dmm/talks/ncrg88.pdf 25

Transformation à

Said Another Way: Open Source has Transformed the Good-Cheap-Fast Development Cycle

26

Fast Fast

Fast

Why? Because you can build Good or Cheap from Fast by using OS Development methodologies and leveraging the OS communities (this is a form of leveraged Investment)

•  Transparency ma\ers

• When there are disagreements in the community •  Transparency makes everyone feel heard •  Transparency makes sure the community does not fracture

• OpenDaylight is transparent to the extreme •  Calls, mailing lists, wikis… are open to anyone •  Even the technical steering commi\ee calls

Transparency

27


Neutron Update •  5+ projects wan>ng to do Neutron integra>on

•  Includes VTN, Dove, GBP, OVSDB, LISP Flow Mapping, VPN Service, …

• What’s missing?

• Clearly need an abstrac>on for projects wan>ng to use Neutron

Neutron-ODL Stack Evolution -- Proposal

OpenStack

Neutron (REST)

OVS

Current World

OpenFlow + OVSDB + Nicira Extensions

OpenStack

Neutron (REST)

Tunnel Mgmt, Traffic Dir, Flow Prgmming, etc…

REST/YANGAdapter

Higher-Level Network Virtualization API

App

Neutron (YANG)

Neutron ODL Policy Adapter App

Tnnl Mgmt ??? ???

Generic Overlay Network ???

Possible Future API Layers

Many h/w- and v-switches

OVSDB NETCONF SNMP OpenFlow+Nicira Extns?

Relevant Southbound Protocol

OpenStack

Neutron (REST)

Tunnel Mgmt, Traffic Dir, Flow Prgmming, etc…


Long-Term Strategy

REST/YANGAdapter


App

Neutron (YANG)


Various new drivers/providers and layering over time as we learn what makes sense

OVSDB NETCONF SNMP OpenFlow+Nicira Extns?


OpenStack

Neutron (REST)

Tunnel Mgmt


Near Future

REST/YANGAdapter

Neutron (YANG)

$(NEW NAME FOR OVSDB NETVIRT)Network virtualization layer that is still “hard-wired” to

Neutron above, but now uses more general APIs below.

OVSDBPlugin

NETCONFIf developers

show up to help

OpenFlow+Nicira Extns?


OpenStack

Neutron (REST)


Lithium Release?

REST/YANGAdapter


App

Neutron (YANG)


$(NEW NAME FOR OVSDB NETVIRT)Network virtualization layer that now uses the more

general APIs above and below.


OVSDBThis project is a

monolithic combination of:(1) a network

virtualization layer that is “hard-wired” to Neutron above

and OVS below as well as (2) an

OVSDB protocol library.

Traffic Direction

Major needed work:

1.) Factor apart two halves of the OVSDB project—network virt. layer and OVSDB library.

2.) Migrate network virt. layer to use the MD-SAL.

3.) Turn OVSDB protocol library into a plugin—not just a library.

4.) Migrate OVSDB protocol plugin to use the MD-SAL.

5.) Close feature gap between network virt. layer and OVS OS plugin, e.g., FWaaS, VPNaaS.

6.) Test/improve scale, stability, and performance of the stack.

7.) Migrate network virt. layer’s NB API from Neutron (REST) to Neutron (YANG) to Policy/Intent.

8.) Migrate network virt. layer’s SB APIs from OVS-specific to tunnel management and traffic direction (into tunnels). Tunnel Mgmt

OVSDBPlugin NETCONF OpenFlow

+Nicira Extns?

Traffic Direction

Slide courtesy Chris Wright and Colin Dixon

•  Con>nue to build/refine our community •  Including increasing commi\er diversity within and across the projects •  Code Quality and Coverage

•  Stability, Security, Performance, Bug fixes ($Major.$Minor) •  Distributed Systems Issues •  S3P – Stability, Scalability, Security, and Performance

•  “Staffing” •  Release engineering •  Tes>ng and Integra>on •  Documenta>on •  …

•  Con>nue to refine our engineering systems •  Thanks Linux Founda>on!

•  We need more code that writes code

•  MD-‐SAL is an example •  Fewer humans in the loop •  More automa>on more be\er

Quasi-technical things we’re working on (necessarily incomplete list)

31


www.opendaylight.org

§  Pull code and try it out

§  TSC weekly calls open to everyone

▪  http://wiki.opendaylight.org

▪  Keep informed and join the conversation ▪  IRC: #opendaylight on irc.freenode.net ▪  Email: lists.opendaylight.org ▪  Facebook: @openDaylightSDN ▪  Twitter: #OpenDaylight

Get Involved!

33

Q&A

Thanks!

Documents

Introduction to OpenDaylight: Current Events and OpenStack Neutron ...dmm/talks/2015/openstack_austin.pdf · Introduction to OpenDaylight: Current Events and OpenStack Neutron Integration