©  MIRANTIS  2012   PAGE  1  ©  MIRANTIS  2012  

Does Hypervisor Matter in OpenStack

Greg Elkinbard Senior Technical Director

©  MIRANTIS  2012   PAGE  2  

Outline

• Brief History of Time: Hypervisor Requests •  2011, 2012, 2013

• Trends in Different Segments

• Opportunities and Challenges

©  MIRANTIS  2012   PAGE  3  

2011

• Hypervisors •  XEN

•  Default choice given Rackspace and Amazon use

•  KVM •  Bleeding edge users

©  MIRANTIS  2012   PAGE  4  

2012

• Hypervisors •  KVM

•  Emerges as the lead

•  XEN •  Loses momentum

©  MIRANTIS  2012   PAGE  5  

2013

• Hypervisors •  KVM

•  Maintains lead (around 90%+ for Mirantis)

•  Vmware •  Emerges as a surprising second choice

•  Containers (LXC, Parallels, Docker) •  Web Hosting and SAS focused

•  Xen and HyperV •  Infrequent requests

©  MIRANTIS  2012   PAGE  6  

Trends

•  Telco/ISP public clouds •  Single hypervisor •  KVM

•  Internet focused companies •  Single hypervisor •  KVM

•  Web Hosting and SAS sub-segments •  Single or multi hypervisors •  Containers (LXC, Parallels, Docker) •  KVM

•  Enterprise •  Multi hypervisors •  KVM and vCenter/ESXi

©  MIRANTIS  2012   PAGE  7  

Opportunities and Challenges

• Multi-hypervisor use cases

• Advantages and Issues •  KVM •  vCenter/ESXi •  Containers (LXC, Docker, Parallels)

©  MIRANTIS  2012   PAGE  8  

Multi Hypervisor Use Cases

• Existing virtualization infrastructure •  Extend instead of replace

• Provide a common API

• Hedge bets against bugs and vendor pricing

• Utilize additional features •  Virtual appliances not supported by other platforms •  Advanced features not directly supported by OpenStack

©  MIRANTIS  2012   PAGE  9  

KVM

• Currently about 90%+ of requests, 95%+ of deployments • Type 2 hypervisor •  Relies on a distro •  Relatively easy to add new devices •  Easy to tune to get good performance

•  Flexibility creates speedup opportunities •  Mellanox e-switch •  Intel DPDK vswitch

•  Few issues •  Mostly distro related

©  MIRANTIS  2012   PAGE  10  

KVM Tuning

• We use KVM for HPC and NFV • Extensive set of tunings • Short Summary •  BIOS to max performance •  Enable huge pages •  On RHEL tune for virtualization •  Libvirt configure host pass-through for CPU flags •  Increase TCP buffers and processor input queue •  Congestion control to htcp •  Enable jumbo frames

©  MIRANTIS  2012   PAGE  11  

KVM performance results

• KVM with OVS •  With tuning 7-8 Gbits/s VM throughput (jumbo frames)

• KVM with Mellanox •  Throughput host to host: 19.4 Gbit/s •  VM to VM on same host: 13.9 Gbit/s •  VM to VM on different hosts: 23.1 Gbit/s

• KVM with Intel OVS DPDK •  Alternative to dedicated hardware •  Good performance even for small packets

•  7Gbit/s for 64 byte packets •  9.6Gbit/s for 512 byte packets

©  MIRANTIS  2012   PAGE  12  

KVM Features and Issues

• Widest set of OpenStack Features •  https://wiki.openstack.org/wiki/HypervisorSupportMatrix

•  Issues •  Difficult to transfer images from other hypervisors •  Older versions of QEMU in RHEL/CentOS require virtio

drivers •  SCSI emulation in Ubuntu •  Updated CentOS with latest QEMU

©  MIRANTIS  2012   PAGE  13  

VMware ESXi

• Currently about 8-10% of requests, 5%+ of deployments •  Primarily vCenter

• Type 1 hypervisor •  Does not need a linux distro •  VMware controls the code, 3rd party packages must be

signed.

• VMware is supporting both ESXi and vCenter APIs

©  MIRANTIS  2012   PAGE  14  

VMware OpenStack Compatibility

• Good compatibility with OpenStack Features •  https://wiki.openstack.org/wiki/HypervisorSupportMatrix •  Pause, Un-Pause and Resize missing

• Caveats (Grizzly) •  Network integration

•  Nova-net no security groups •  Neutron requires Nicira commercial plugin

•  Glance integration inefficient •  Only a single datastore is supported •  With Cinder only iSCSI type volumes are supported •  Only linked clones are supported

©  MIRANTIS  2012   PAGE  15  

Vmware OpenStack Havana

• Linked and Full clones are supported

• Multiple VC clusters can be managed by a single driver

• Config drive support

• Cinder support for VMDK based Volumes

• vShield Edge Driver support for NVP plugin (FWaaS, LBaaS)

©  MIRANTIS  2012   PAGE  16  

VMware Networking

• Nicira NVP/NSX

• Cisco 1000V (future)

• Accelerated Options •  NVP using STT •  VN-Tag, SR-IOV & Cisco Switches (VM-FEX)

©  MIRANTIS  2012   PAGE  17  

Containers

• Currently about 5%+ of requests, 0 deployments

•  Low overhead alternative •  Hundreds of guests •  All Apps belong to a single tenant

•  Fragmented space •  LXC •  Parallels •  Docker

•  Limited OpenStack support but interest is growing

©  MIRANTIS  2012   PAGE  18  

LXC OpenStack Support

• VM – only launch, reboot and terminate

• Networking – basic VLANs •  Possible to get Neutron/OVS to work

• Volumes – officially no Cinder support •  Possible to hack something