I/O virtualization with InfiniBand and 40 Gigabit Ethernet

Ali Ayoub – Staff Architect

VMworld 2013 San Francisco

© 2013 Mellanox Technologies 2

Remote Direct Memory Access

What is RDMA?

RDMA

Engine

Ethernet (RoCE)

InfiniBand

© 2013 Mellanox Technologies 3

Virtual Protocol Interconnect Technology

VPI Eth

InfiniBand

What is VPI?

© 2013 Mellanox Technologies 4

IPoIB

iSER/SRP

iSER • 10GigE

• 40GigE VP

I E

th

Infi

niB

an

d

InfiniB

and

Eth

ern

et

Mellanox Software Solutions

© 2013 Mellanox Technologies 5

Mellanox Software Solutions

RDMA

VPI

TCP/IP

10GigE

40GigE

iSCSI

NFS

NFS

SRP

iSER

IPoIB

TCP/IP Storage

Solutions

RDMA Storage

Solutions

Network

Solutions

Ethernet

InfiniBand

© 2013 Mellanox Technologies 6

Mellanox Software Solutions

10GigE

40GigE

IPoIB

SRP

iSER

iSCSI

NFS

NFS

RDMA Storage

Solutions

TCP/IP Storage

Solutions

Network

Solutions

CPU Utilization

Bandwidth

Latency

Consolidation

Performance

More Virtual

Machine

© 2013 Mellanox Technologies 7

Mellanox Software Solutions

10GigE

40GigE

IPoIB

SRP

iSER

RDMA Storage

Solutions

Network

Solutions vSphere 5.5 In-box

Part of ESX Distribution

Downloadable from vmware.com

vSphere 5.5 PVSP

Partner Verified and Supported Products

Downloadable from mellanox.com

© 2013 Mellanox Technologies 8

40GigE Performance Results

36Gbps I/O

Bandwidth on

vSphere 5.x !

© 2013 Mellanox Technologies 9

Single VM Migration Acceleration

0.00

5.00

10.00

15.00

20.00

Tim

e (

se

c)

Migration of Passive VM*

10GE 40GE

0.00

10.00

20.00

30.00

40.00

50.00

Tim

e (

se

c)

Migration of Active VM*

10GE C 40GE C

>200%

2X to 3X Faster Execution Time over 40GigE

>350%

One VM Migration with VMware vMotion over

10/40GigE NIC I/O Controller

* 16GB Memory

© 2013 Mellanox Technologies 10

Multiple VM Migration Acceleration

Enabling Concurrent Migration of More VMs over 40GigE

vMotion Average Migration Duration

Du

rati

on

[m

m:s

s]

# VM

Multiple VM Migration with VMware vMotion over

10/40GigE NIC I/O Controller

© 2013 Mellanox Technologies 11

iSER Performance Results

190 IOPS on

vSphere 5

174

176

178

180

182

184

186

188

190

192

2 4 6 8

#VMs/LUNs (LUN per VM)

IOPS

35.2

35.3

35.4

35.5

35.6

35.7

35.8

35.9

36

36.1

2 4 6 8

#VMs/LUNs (LUN per VM)

Bandwidth (Gbps)

© 2013 Mellanox Technologies 12

SRIOV

© 2013 Mellanox Technologies 13

IPoIB

© 2013 Mellanox Technologies 14

iSER/SRP

© 2013 Mellanox Technologies 15

Leading Supplier of End-to-End Interconnect Solutions

Virtual Protocol Interconnect

Storage Front / Back-End

Server / Compute Switch / Gateway

56G IB & FCoIB 56G InfiniBand

10/40/56GbE & FCoE 10/40/56GbE

Virtual Protocol Interconnect

Host/Fabric Software ICs Switches/Gateways Adapter Cards Cables

Comprehensive End-to-End InfiniBand and Ethernet Portfolio

© 2013 Mellanox Technologies

Mellanox 40GbE In-Box with VMware vSphere 5.5

© 2013 Mellanox Technologies