Embed Size (px)
DESCRIPTION
IPoIB. Vivek Kashyap [email protected]. IPoIB Draft Status. IP over InfiniBand Architecture draft-ietf-ipoib-architecture-04.txt Approved to be published as an Informational RFC. Transmission of IP over InfiniBand draft-ietf-ipoib-over-infiniband-06.txt - PowerPoint PPT Presentation
Citation preview
IETF-60: IPoIB Connected Mode
Vivek Kashyap, IBM2
IPoIB Draft Status
IETF-60: IPoIB Connected Mode
Vivek Kashyap, IBM3
IP over InfiniBand Architecture – draft-ietf-ipoib-architecture-04.txt
– Approved to be published as an Informational RFC.
Transmission of IP over InfiniBand– draft-ietf-ipoib-over-infiniband-06.txt
– Issue: EUI64 -> GUID -> Interface Identifier
– Sense of ‘u’ bit– Awaiting IB version 1.2 specification – Resubmit for ‘last-call’
DHCP over InfiniBand– draft-ietf-ipoib-dhcp-over-infiniband-06.txt
– IESG Review
IETF-60: IPoIB Connected Mode
Vivek Kashyap, IBM4
IPoIB Connected Mode
IETF-60: IPoIB Connected Mode
Vivek Kashyap, IBM5
IP over InfiniBand Modes
Unreliable Datagram
Connected– Reliable
– Unreliable
Raw
IETF-60: IPoIB Connected Mode
Vivek Kashyap, IBM6
IPoIB Connected Mode
Large MTU
Automatic Path Migration
Reliability
RDMA
RC
2^31 bytes 2-4K links
Yes
Yes
Yes
UC
2^31 bytes 2-4K links
Yes
No
RDMA Write
IETF-60: IPoIB Connected Mode
Vivek Kashyap, IBM7
IPoIB Subnet IPoIB-Connected subnet is formed over a Non-Broadcast Multiple
Access networkIPoIB-UD is over an abstract broadcast link
Node D
IPd IP
s
Node A A
IPa IPp
IPr IPcNode C
IPq IP
bN
ode B
IB switched network
IPoIB UD broadcast domain
IPoIB Connected network
IETF-60: IPoIB Connected Mode
Vivek Kashyap, IBM8
Address Resolution
IPb IPa
IPoIB UD broadcast/multicast domain
- IPb broadcasts query
- IPa responds UD unicast
IB switched network
- CM REQ message
- CM REP message
- CM RTU message
IPoIB Connected network
- IP communication over connected mode
IETF-60: IPoIB Connected Mode
Vivek Kashyap, IBM9
Address Resolution
IPb: Who Has IPa?– Address resolution over a UD QP
– May reuse IPoIB-UD QP– Use multicast-GID
– FF1x:<401B>:<P_Key>::<255.255.255.255 or 1>– FF1x:<601B>:<P_Key>::<solicited node address>– P_Key of connected network
– [RC|UC]:<QPN>:GIDb – RC|UC indicate the requesters view of the subnet
– IPa receives the packet
– Determine PathRecord to GIDb – Send reply over UD-QP with GIDa
[RC|UC]:<QPN>:GIDa
The UD QPN is used to create the Service ID for setting IB connection
IETF-60: IPoIB Connected Mode
Vivek Kashyap, IBM10
Connection Setup
Connection is setup using IB MADs Service ID
Private Data field– Desired MTU
– Minimum/Accepted MTU
0x1 Resvd3-octetQPN
Resvd ResvdResvd
IETF-60: IPoIB Connected Mode
Vivek Kashyap, IBM11
MTU Negotiation per IB link
A requests MTU in private data of CM REQ message to peer B– Desired MTU in private data
B accepts value in CM REP message to peer A– Accepts value in the range ‘Minimum MTU’ and ‘Desired MTU’
– Respond with ‘Accepted MTU’
A accepts or,Rejects and tries again
Allows a fallback to ‘Minimum MTU’ which is the MTU derived from Broadcast-GID’s MTU
IETF-60: IPoIB Connected Mode
Vivek Kashyap, IBM12
Encapsulation
IP packets encapsulated with the ‘Ethertype’ indication as in IPoIB-UD
Address resolution over UD QP needs to follow IPoIB-UD encapsulation
IETF-60: IPoIB Connected Mode
Vivek Kashyap, IBM13
Ongoing discussions
Multiple connected mode links between peers– Draft says ‘SHOULD NOT’
– Proposal:
Add more detail to draft Multiple IB connects are allowed. But ‘user beware’.
Mixed UC/RC interconnects– Link type determined based on advertisement
– Issue: interoperability
