Openstack Testbed_ovs_virtualbox_devstack_single node

2014.08.14.

신용윤(ETRI, SDN - Virtual Network)

OpenIRIS - iNaaSTestbed 구성

Network Virtualization SDN Controller

실험망 구성Single Server

Using OVS to DevStack

OpenIRIS - iNaaS : The SDN Controller for Network Virtualization

• I have no OpenFlow Switch I can’t use SDN Controller

SDN Controller 는 OF SW …를 제어하는데

• Just private testbed I want very simple test

…개인적으로 간단하게 구성해서 테스트 하고 싶은데

• How can I use OF SW It’s OpenvSwitch

OS 는 ubuntu …로 정해야 되겠군

Single Server 를 사용하는 이유Single Server 실험망


• Server Ubuntu 14.04 LTS

OpenvSwitch 를 설치하여야 하기 때문

Eclipse Java 1.7 OpenIRIS

VirtualBox OpenvSwitch

OpenFlow Enable Switch 대체

물리망을 대체하기 위함

시스템 구성Single Server 실험망


구성도Single Server 실험망

Ubuntu 14.04

ovs-2

ovs-1 ovs-3

openvswitch

virtualbox

Control/Net-workNode

Compute Node #01

Compute Node #02

eth1eth0 eth1eth0 eth1eth0

OpenIRIS

OVS OVS OVS

ovs-4

Compute Node #03

eth1eth0

OVS

P1-2

vnet1-1

vnet2-1

vnet3-1

vnet4-1

P2-1

P2-3

P3-2

P3-4

P4-3


• OS Install Ubuntu 12.04 or 14.04 LTS desktop version

Using CD img• Ubuntu configuration

Network

OS update

Single Server OS 설치Single Server 실험망

$ sudo apt-get purge network-manager$ sudo apt-get autoremove

$ sudo vi /etc/network/interfaces

$ sudo /etc/init.d/networking restart

$ sudo apt-get update$ sudo apt-get upgrade$ sudo apt-get dist-upgrade

$ sudo reboot

auto eth0Iface eth0 inet static address <your public ip> netmask 255.255.255.0 gateway <your gw ip> dns-nameservers 8.8.8.8 8.8.4.4


• Must Install and Downloads git/vim/java1.7 Install OpenIRIS download Download Eclipse

http://eclipse.org/downloads/?osType=linux&release=undefined 압축해제

Download Ubuntu Img Using VirtualBox VM (DevStack install)

Single Server 필수 프로그램 설치Single Server 실험망

$ sudo apt-get –y git$ sudo apt-get –y vim$ sudo apt-get –y openjdk-7-jeadless

$ git clone https://github.com/bjlee72/IRIS.git


• OpenvSwitch Download 2.0.1 Version http://openvswitch.org/pipermail/announce/2014/March/0000059.html

Single Server OpenvSwitch 설치 (1/2)

Single Server 실험망

http://openvswitch.org/pipermail/announce/2014/March/0000059.html

http://openvswitch.org/pipermail/announce/2014/March/0000059.html


• OpenvSwitch Install Must Read : openvswitch 디렉토리 (INSTALL.Debian)

Single Server OpenvSwitch 설치 (2/2)


$ cd openvswitch-2.0.1

$ sudo apt-get install build-essential fakeroot# dpkg-checkbuilddeps 명령을 통하여 dependency 내용을 확인할 수 있다 . $ sudo apt-get install debhelper autoconf automake libssl-dev$ sudo apt-get install python-all python-qt4 python-twisted-conch$ fackroot debian/rules binary# home directory 에 .deb 파일들이 생성된다 . $ cd~$ sudo apt-get install dkms$ sudo dpkg –i openvswitch-datapath-dkms_2.0.1.1_all.deb$ sudo apt-get install module-assistant$ sudo dpkg –i openvsiwtch-datapath-source_2.0.1-1_all.deb$ sudo dpkg –i openvswitch-common_2.0.1-1_amd64.deb$ sudo dpkg –i openvswitch-switch_2.0.1-1_amd64.deb


• VirtualBox Download and install Ubuntu Software Center Find : virtualbox

Single Server VirtualBox 설치Single Server 실험망


• OpenvSwitch Configuration Make 4 Bridges : ovs-1, ovs-1, ovs-3, ovs-4

Single Server OpenvSwitch 설정 (1/5)


$ sudo ovs-vsctl add-br ovs-1$ sudo ovs-vsctl add-br ovs-2$ sudo ovs-vsctl add-br ovs-3$ sudo ovs-vsctl add-br ovs-4


• OpenvSwitch Configuration Make tuntap : vnet1-1, vnet2-1, vnet3-1, vnet4-1



$ sudo ip tuntap add mode tap vnet1-1$ sudo ip tuntap add mode tap vnet2-1$ sudo ip tuntap add mode tap vnet3-1$ sudo ip tuntap add mode tap vnet4-1$ sudo ip link set vnet1-1 up$ sudo ip link set vnet2-1 up$ sudo ip link set vnet3-1 up$ sudo ip link set vnet4-1 up


• OpenvSwitch Configuration Connect Bridge to tuntap : ovs-1:vnet1-1, ovs-2:vnet2-1, ovs-3:vnet3-1, ovs-4:vnet4-1



$ sudo ovs-vsctl add-port ovs-1 vnet1-1$ sudo ovs-vsctl add-port ovs-1 vnet2-1$ sudo ovs-vsctl add-port ovs-1 vnet3-1$ sudo ovs-vsctl add-port ovs-1 vnet4-1


• OpenvSwitch Configuration Make patch : p1-2, p2-1, p2-3, p3-2, p3-4, p4-3



$ sudo ovs-vsctl add-port ovs-1 p1-2 -- set Interface p1-2 type=patch options:peer=p2-1$ sudo ovs-vsctl add-port ovs-2 p2-1 -- set Interface p2-1 type=patch options:peer=p1-2$ sudo ovs-vsctl add-port ovs-2 p2-3 -- set Interface p2-3 type=patch options:peer=p3-2$ sudo ovs-vsctl add-port ovs-3 p3-2 -- set Interface p3-2 type=patch options:peer=p2-3$ sudo ovs-vsctl add-port ovs-3 p3-4 -- set Interface p3-4 type=patch options:peer=p4-3$ sudo ovs-vsctl add-port ovs-4 p4-3 -- set Interface p4-3 type=patch options:peer=p3-4


• OpenvSwitch Configuration Conntction OpenIRIS : ovs-1, ovs-2, ovs-3, ovs-4



$ sudo ovs-vsctl set-controller ovs-1 tcp:<your public ip>:6633$ sudo ovs-vsctl set-controller ovs-2 tcp:<your public ip>:6633$ sudo ovs-vsctl set-controller ovs-3 tcp:<your public ip>:6633$ sudo ovs-vsctl set-controller ovs-4 tcp:<your public ip>:6633


• VirtualBox Basic Configuration Make 4 VMs : controlnode, com01, com02, com03, com04

Memory Size : over 4G Hard Drive Size : over 32G

Single Server VirtualBox 설정 (1/3)



• VirtualBox Network Configuration VMs Network Setting

eth0 : NAT / eth1 : Bridge Adapter (tuntap select) Warning

controlnede – vnet1-1 / com01 – vnet2-1 / com02 – vnet3-1 / com03 – vnet4-1



com01 -> vnet2-1com02 -> vnet3-1com03 -> vnet4-1


• VirtualBox Configuration Complete You can have OpenFlow Switchs

ovs-1, ovs-2, ovs-3, ovs-4 You can have Hosts

controlnede / com01 / com02 / com03



실험망 구성DevStack Install


• controlnode OS Install : ubuntu 14.04

Already downloaded ubuntu

Single Server DevStack Install (1/10)



• controlnode Must Configuration



$ sudo vi /etc/network/interfaces auto loiface lo inet loopback # Public Network, NATauto eth0iface eth0 inet static address <NAT IP> # may be default is 10.0.2.15 netmask 255.255.255.0 gateway <GW IP> # may be default is 10.0.2.2 dns-nameservers 8.8.8.8 8.8.4.4 # Data Network, Private IPauto eth1iface eth1 inet static address <private ip> netmaks 255.255.255.0 $ sudo apt-get purge network-manager$ sudo apt-get autoremove$ sudo /etc/init.d/networking restart


• controlnode Must Configuration

DevStack Download



$ sudo apt-get update$ sudo apt-get upgrade$ sudo apt-get dist-upgrade$ sudo reboot

$ sudo su$ echo “stack ALL=(ALL:ALL) NOPASSWD:ALL” >> /etc/sudoers

$ sudo vi /etc/hosts …<controlnode private IP> controlnode<com01 private IP> com01<com02 private IP> com02<com03 private IP> com03…

$ sudo apt-get install –y git vim

$ git clone –b stable/icehouse https://github.com/openstack-dev/devstack.git


• controlnode DevStack Download



$ cd devstack$ vim local.conf [[local|localrc]]GIT_BASE=${GIT_BASE:-https://git.openstack.org} # CredentialsDATABASE_PASSWORD=openstackADMIN_PASSWORD=openstackSERVICE_PASSWORD=openstack SERVICE_TOKEN=admin RABBIT_PASSWORD=openstack # ServicesENABLED_SERVICES=rabbit,mysql,keyENABLED_SERVICES+=,n-api,n-crt,n-obj,n-cond,n-sch,n-novnc,n-cauthENABLED_SERVICES+=,neutron,q-svc,q-agt,q-dhcp,q-l3,q-metaENABLED_SERVICES+=,g-api,g-regENABLED_SERVICES+=,cinder,c-api,c-vol,c-sch,c-bakENABLED_SERVICES+=,horizon # MULTI_HOST MULTI_HOST=True # Services IPHOST_IP=<controlnode private ip>

# Use this image when creating test instancesIMAGE_URLS+=",http://cdn.download.cirros-cloud.net/0.3.2/cirros-0.3.2-x86_64-disk.img" # vncVNCSERVER_LISTEN=0.0.0.0VNCSERVER_PROXYCLIENT_ADDRESS=$HOST_IP # BranchesKEYSTONE_BRANCH=stable/icehouseNOVA_BRANCH=stable/icehouseNEUTRON_BRANCH=stable/icehouseSWIFT_BRANCH=stable/icehouseGLANCE_BRANCH=stable/icehouseCINDER_BRANCH=stable/icehouseHEAT_BRANCH=stable/icehouseTROVE_BRANCH=stable/icehouseHORIZON_BRANCH=stable/icehouse # Enable LoggingLOGFILE=/opt/stack/logs/stack.sh.logVERBOSE=TrueLOG_COLOR=TrueSCREEN_LOGDIR=/opt/stack/logs


• controlnode Start DevStack



$ ./stack.sh


• com01, com02, com03 OS Install : ubuntu 14.04

Already downloaded ubuntu




• com01, com02, com03 Must Configuration



$ sudo vi /etc/network/interfaces auto loiface lo inet loopback # Public Network, NATauto eth0iface eth0 inet static address <NAT IP> # may be default is 10.0.2.15 netmask 255.255.255.0 gateway <GW IP> # may be default is 10.0.2.2 dns-nameservers 8.8.8.8 8.8.4.4 # Data Network, Private IPauto eth1iface eth1 inet static address <private ip> netmaks 255.255.255.0 $ sudo apt-get purge network-manager$ sudo apt-get autoremove$ sudo /etc/init.d/networking restart


• com01, com02, com03 Must Configuration

DevStack Download



$ sudo apt-get update$ sudo apt-get upgrade$ sudo apt-get dist-upgrade$ sudo reboot

$ sudo su$ echo “stack ALL=(ALL:ALL) NOPASSWD:ALL” >> /etc/sudoers

$ sudo vi /etc/hosts …<controlnode private IP> controlnode<com01 private IP> com01<com02 private IP> com02<com03 private IP> com03…

$ sudo apt-get install –y git vim

$ git clone –b stable/icehouse https://github.com/openstack-dev/devstack.git


• com01, com02, com03 DevStack Download



$ cd devstack$ vim local.conf [[local|localrc]]DATABASE_PASSWORD=openstackADMIN_PASSWORD=openstackSERVICE_PASSWORD=openstackSERVICE_TOKEN=adminRABBIT_PASSWORD=openstack # ServicesENABLED_SERVICES=rabbitENABLED_SERVICES+=,n-cpu,n-novnc,n-cauthENABLED_SERVICES+=,neutron,q-agt # MULTI_HOSTMULTI_HOST=True # Services IPHOST_IP=<com01 private IP>SERVICE_HOST=<controlnode private IP>MYSQL_HOST=<controlnode private IP>RABBIT_HOST=<controlnode private IP>GLANCE_HOSTPORT=<controlnode private IP>:9292Q_HOST=<controlnode private IP>

# Neutron OVS (vxlan)Q_PLUGIN=ml2Q_ML2_TENANT_NETWORK_TYPE=vxlanQ_ML2_PLUGIN_TYPE_DRIVERS=vxlanQ_ML2_PLUGIN_VXLAN_TYPE_OPTIONS=(vni_ranges=1001:2000)Q_AGENT_EXTRA_AGENT_OPTS=(tunnel_types=vxlan vxlan_udp_port=8472)Q_USE_NAMESPACE=TrueQ_USE_SECGROUP=True # vncVNCSERVER_LISTEN=0.0.0.0VNCSERVER_PROXYCLIENT_ADDRESS=$HOST_IP # BranchesKEYSTONE_BRANCH=stable/icehouseNOVA_BRANCH=stable/icehouseNEUTRON_BRANCH=stable/icehouseSWIFT_BRANCH=stable/icehouseGLANCE_BRANCH=stable/icehouseCINDER_BRANCH=stable/icehouseHEAT_BRANCH=stable/icehouseTROVE_BRANCH=stable/icehouseHORIZON_BRANCH=stable/icehouse # Enable LoggingLOGFILE=/opt/stack/logs/stack.sh.logVERBOSE=TrueLOG_COLOR=TrueSCREEN_LOGDIR=/opt/stack/logs


• controlnode Start DevStack



$ ./stack.sh

Test


• OpenIRIS to OpenvSwitch OpenIRIS connection to OVS (ovs-1, ovs-2, ovs-3, ovs-4)

Run OpenIRIS using Eclipse You must edit ‘torpedo.properties’ SimpleOFController = false BasicOFController = true

Single Server Test


Connected OVS -> ovs-1, ovs-2, ovs-3, ovs-4

OpenIRIS con-sole


• OpenIRIS to OpenvSwitch Web page

Single Server ip:8080

Single Server Test


ovs-1, ovs-2, ovs-3, ovs-4

controlnode, com01, com02, com03

ovs-1, ovs-2, ovs-3, ovs-4

controlnode, com01, com02, com03


• DevStack (create Network and VM) Create Network (2 Networks)

Net01 : 30.0.0.0/24, Net02 : 40.0.0.0/24 Create VM (4VMs)

Net01: vm01, vm03 / Net02: vm02, vm04

Single Server Test



• DevStack (Node OVS) networknode

Net01 : 30.0.0.0/24, Net02 : 40.0.0.0/24 com01, com02, com03

Net01: vm01, vm03 / Net02: vm02, vm04

Single Server Test


com01 : create vm02 port

vxlan configurationvxlan configuration

controlnode : Net01, Net02

controlnode : sudo ovs-vsctl show com01 : sudo ovs-vsctl show


• DevStack (Ping) com01, com02, com03

Net01: vm01, vm03 / Net02: vm02, vm04 Ping : vm01 to vm03 (It’s same subnet)

Single Server Test


vm01, vm03 Same Network : con-nected


• DevStack (Ping) com01, com02, com03

Net01: vm01, vm03 / Net02: vm02, vm04 Ping : vm01 to vm02 (It’s different subnet)

Single Server Test


vm01, vm02 Different Network : Not connected


• DevStack (create Virtual Router) Connected Net01 to Net02 : Router01

Net01: vm01, vm03 <-> Router01 <-> Net02: vm02, vm04

Single Server Test



• DevStack (create Router Node OVS) networknode

Virtual Router Routing Net01 to Net02

Single Server Test


controlnode : virtual router port Net01, Net02

controlnode : sudo ovs-vsctl show


• DevStack (check Virtual Router) Connected Net01 to Net02 : Router01

Net01: vm01, vm03 <-> Router01 <-> Net02: vm02, vm04

Single Server Test



• DevStack (Ping) com01, com02, com03 Connected Net01 to Net02 : Router01

Net01: vm01, vm03 <-> Router01 <-> Net02: vm02, vm04 Ping : vm01 to vm02 (It’s different subnet)

Single Server Test


vm01, vm02 Different Network : Connected (Router01)

TodoMultiNode Testbed


• OpenIRIS Controller Ubuntu 14.04 server/cloud LAN port 3

• Control Node Ubuntu 14.04 LTS or Ubuntu 14.04 server/cloud LAN port 2 For Migration

• Network Node Ubuntu 14.04 server/cloud LAN port 3

• Compute Node (over 2 systems) Ubuntu 14.04 server/cloud LAN port 2

• Switch (OpenFlow SW : 6 ea, SW : 3 ea) OpenFlow Switch (Data Network) Switch (Public Network, Management Network, Aggregation)

Todo Multi Node

Multi Nodes 실험망


• NEC PF5240(2ea) Hardware

Interface : 1GbE(1000base-T) 48 port 10G Ethernet(SFP) 4port

Switch Capacity 176Gbps switching capacity

OpenFlow Mode Func. OpenFlow 1.0 160,000 Flow entries

L2/L3 Mode Func. L3: OSPF, BGP, RIP, VRRP, PIM, MLD L2: STP, RSTP, MSTP, PVST, IGMP

OpenFlow Switch Spec. (1/2, NEC PF5240)



• NetVision (4ea) Hardware

Interface : 1GbE(1000base-T) 48 port 10G Ethernet(SFP) 4port

Switch Capacity 132Gbps switching capacity Broadcom switch chip

OpenFlow Mode Func. OpenFlow 1.0/1.3 OF-Config 1.0 2048 Flow tables

L2/L3 Mode Func. RIP/OSPF routing protocol PIM-SM multicast routing

OpenFlow Switch Spec. (2/2, NetVision)



간략 구성도Multi Nodes 실험망

Control Node

OpenIRIS Network NodeCompute Node

#01Compute Node

#02

eth1

eth0

eth1

eth0

eth1

eth0

eth2:1

eth0 eth1:

1

eth0 eth1:

1

물리 NIC

OVS

Management Net-work

(IP: 10.0.0.0/24)

Public Network(IP: 129.254.197.0/254,

GW: 129.254.197.1)

Management Net

129.254.197.184, external

129.254.197.183

129.254.197.182

Data Network(IP: 10.0.10.0/24,

OpenFlow)

가상 연결

10.0.0.11management

10.0.0.21management

10.0.0.31management

10.0.0.41management

10.0.0.51management10.0.10.31

Instance tun-nel

10.0.10.41Instance tun-

nel

10.0.10.51Instance tun-

nel

eth2 eth1 eth1eth210.0.10.2

1Data Net

Data Net


세부 구성도Multi Nodes 실험망

Data Network(IP:

10.0.10.0/24)

OpenIRISNetwork

Node

Con-trol

Node

OVSOVS OVSOVS OVSOVS OVS

eth2

eth2

eth2

eth2

eth2

eth2

eth2

eth2

eth0

VM

eth1

eth1

eth1

eth1

eth1

eth1

eth1

eth1

eth1

VM

VM

VM

VM

VM

VM

VM

VM

VM

VM

VM

VM

VM

VM

VM

VM

VM

VM

VM

VM

VM

VM

VM

VM

Management Net-work

(IP: 10.0.0.0/24)

eth1

eth0

eth2

Public Net-work

eth0

Compute Nodes (#01 ~ #n)

NEC#02

Netvision#03

Netvision#04

Aggregation SW

NEC#01

exter-nal

OVS

VM

VM

VM

Netvision#01

Netvision#02

TodoMechanism Driver

For OpenIRIS &OpenIRIS Module


• OpenStack plugin ML2 plugin

Mechanism Driver for OpenIRIS : v0.1 dev. Complete Network REST : create_network, update_network, delete_network Subnet REST : create_subnet, update_subnet, delete_subnet Port REST : create_port, update_port, delete_port Now testing

Mechanism Driver for OpenIRIS v1.0 OVS Bridge and Port create We want managed OVS from OpenIRIS Sync. (Network, Subnet, Port, … info.) We want managed network type(gre, vxlan, …) from OpenIRIS

Develop Mechanism Driver

Mechanism Driver


• OpenIRIS Module ML2 Manager

REST call from mechanism_iris (openstack plugin) Now testing

Tunnel Manager Create Port to OVS using Network type (gre, vxlan, …) Support overlay network

L3 Routing Manager Support L3 Router (ex: virtual router)

…

Develop OpenIRIS Module

Mechanism Driver

Technology

Openstack Testbed_ovs_virtualbox_devstack_single node