of 21/21
Mellanox VXLAN offload for RHEL7 株式会社アルティマ Nov, 2014 本資料に含まれる測定データは一例であり、測定構成や条件によって変わることがあります。 また、本資料はMellanox Technologies社の公式見解を表すものではありません。 The results in this documents may differ for the configurations or/and conditions. This documents does not reflect the official views of Mellanox Technologies.

Mellanox Vxlan offload Performance on linux

  • View
    860

  • Download
    10

Embed Size (px)

DESCRIPTION

Mellanox社製 10/40GENIC(ConnectX-3 Pro)を使ったVXLANオフロードの性能評価をLinux環境で実施しました。 VXLANプロトコルを使うとソフトウェア処理が大きくなるので、オフロードはMust!!

Text of Mellanox Vxlan offload Performance on linux

  • 1. Mellanox VXLAN offloadfor RHEL7Nov, 2014Mellanox TechnologiesThe results in this documents may differ for the configurations or/and conditions.This documents does not reflect the official views of Mellanox Technologies.

2. 2 Mellanox // [email protected] 3. Linux OpenvSwitch(OVS) 10/40/56GbENICVXLAN offloadOVS Linux OVS(Open vSwitch)VXLAN Case1 : Performance VM to VM Case2 : VM to VM VXLAN offload Case3 : VM to VM VXLAN offload 3ConnectX-3 Pro EN 10/40/56GbE adapter cardshttp://www.mellanox.com/page/products_dyn?product_family=162 4. Requirement Mellanox ConnectX-3 Pro Operating system and kernel options: upstream Linux 3.14 or later RHEL7 beta snapshot 10 (kernel 3.10.0-105.el7) or later Ubuntu 14.04 (kernel 3.13.0-24-generic) or later openvswitch 2.0 or later KVM Hypervisor using para-virtual NIC (e.g. virtio withvhost backend on the hypervisor) MLNX_OFED (2.2 or later) installation is optional, asdriver support is inbox (RHEL 7 or Ubuntu 14.04)http://community.mellanox.com/docs/DOC-14464 5. 5 CPU : 12core Xeon 2.40GHz (Fujitsu RX200S8)VM CentOS7CentOS7 VMOVS 2.0.040GE Driver MEM : 18GB OS : CentOS7 (Kernel : 3.10.0-123.el7.x86_64) Driver : MLNX_OFED_LINUX-2.3-1.0.1-rhel7.0-x86_64.iso OVS : ver 2.0 Mellanox 10/40GbE NIC : ConnectX-3 Pro (FW:2.32.5100)OVS 2.0.040GE DriverVM infoOS:CentOS6.5vCPU : 4corevMEM : 4GB 6. 6URL : http://community.mellanox.com/docs/DOC-1446 7. KVM Install packagesOpenStackOK VM Manager7# yum install qemu-kvm# yum install qemu-img# yum install virt-manager# yum install virt-install# yum install libvirt-daemon-config-network# virt-managerVM 8. Mellanox OFED http://www.mellanox.com/page/products_dyn?product_family=26&mtag=linux_sw_drivers 8# mount -o ro,loop MLNX_OFED_LINUX-2.3-1.0.1-rhel7.0-x86_64.iso /mnt# ./mnt/mlnxofedinstall# /etc/init.d/openibd restart 9. OVSOVS v2.0.0 Openstack icehouse Linux Bridge9# yum install -y https://repos.fedorapeople.org/repos/openstack/openstack-icehouse/rdo-release-icehouse.rpm# yum install y openvswitch# modprobe openvswitch# lsmod | grep openvopenvswitch 70743 0vxlan 37584 1 openvswitchgre 13808 1 openvswitchlibcrc32c 12644 3 xfs,btrfs,openvswitch# lsmod | grep bridgebridge 110196 0stp 12976 1 bridgellc 14552 2 stp,bridge# rmmod bridge 10. OVS Openvswitch10# systemctl list-unit-files | grep openvswitchopenvswitch-nonetwork.service staticopenvswitch.service disabled# systemctl enable openvswitch# systemctl start openvswitch# systemctl status openvswitchopenvswitch.service - Open vSwitchLoaded: loaded (/usr/lib/systemd/system/openvswitch.service; enabled)Active: active (exited) since Thu 2014-09-18 19:14:42 JST; 2min 20s agoMain PID: 12972 (code=exited, status=0/SUCCESS)Sep 18 19:14:42 ALT1 systemd[1]: Started Open vSwitch.# ovs-vsctl show5c682e27-2498-4188-ac25-269cacedbe7fovs_version: "2.0.0" 11. 11VM1eth0vnet0bridgeovs-vxeth0NICVM2eth0vnet0bridgeovs-vxeth0NICPoint Bridge ovs-vx VM1NIC 12. Bridge & (NICOK) (VM)12# ovs-vsctl add-br ovs-vx# ovs-vsctl add-port ovs-vx enp4s0# virsh edit ) 13. vnet0ovs13[[email protected] ~]# ovs-vsctl showdaaecc1a-5344-4387-9204-399b101d8b8eBridge ovs-vxPort "vnet0"Interface "vnet0"Port "enp4s0"Interface "enp4s0"Port ovs-vxInterface ovs-vxtype: internalovs_version: "2.0.0"[[email protected] ~]#[[email protected] ~]# ovs-dpctl [email protected]:lookups: hit:8320258 missed:41 lost:0flows: 0port 0: ovs-system (internal)port 1: ovs-vx (internal)port 2: enp4s0port 3: vnet0 14. VXLAN offload MLNX OFED 2.3.1VXLAN offload = enable Disable141. Open the /etc/modprobe.d/mlnx.conf file.2. Set the options mlx4_core log_num_mgm_entry_size=10.3. Restart the driver (/etc/init.d/openibd restart) 15. OVS VXLAN OVS # ovs-vsctl del-port ovs-vx port OVSVXLAN50byteIPMTU(MTU=1550) Ex) # ip link set enp4s0 mtu 155015# ovs-vsctl add-br ovs-vx# ovs-vsctl add-port ovs-vx vxlan0 -- set interface vxlan0 type=vxlanoptions:remote_ip=10.0.0.20 options:key=99 options:dst_port=4789 16. OVS VXLAN 16[[email protected] ~]# ovs-vsctl showdaaecc1a-5344-4387-9204-399b101d8b8eBridge ovs-vxPort ovs-vxInterface ovs-vxtype: internalPort "vnet0"Interface "vnet0"Port "vxlan0"Interface "vxlan0"type: vxlanoptions: {dst_port="4789", key="99", remote_ip="10.0.0.20"}ovs_version: "2.0.0[[email protected] ~]# ovs-dpctl [email protected]:lookups: hit:439298 missed:34 lost:0flows: 2port 0: ovs-system (internal)port 1: vxlan_sys_4789 (vxlan: df_default=false, ttl=0)port 2: ovs-vx (internal)port 3: vnet0 17. 17 18. Performance 10GE18Mellanox Technologiesiperf Ver2.0.5 : # iperf -c 10.0.0.101 -P 4MTU=1500 19. Performance 40GE19Mellanox Technologiesiperf Ver2.0.5 : # iperf -c 10.0.0.101 -P 4MTU=1500 20. OVSVXLAN1/7 Mellanox ConnectX-3 ProVXLAN offload5.6 VXLANOffloadCPU20 21. 212SE MAIL : [email protected]