HPE 3PAR StoreServ Persistent Ports technical white paper › V2 › GetPDF.aspx › 4AA4... · Terminology • N_Port ID Virtualization (NPIV): A Fibre Channel (FC) facility that

HPE 3PAR StoreServ Persistent Ports

Contents Terminology................................................................................................................................................................................................................................................................................................................................ 2

Executive summary .............................................................................................................................................................................................................................................................................................................. 2

Features and benefits ........................................................................................................................................................................................................................................................................................................ 2

What’s new with Persistent Ports on HPE 3PAR? ................................................................................................................................................................................................................................... 3

What does it do? ..................................................................................................................................................................................................................................................................................................................... 3

Requirements ............................................................................................................................................................................................................................................................................................................................ 3

Configuration considerations ...................................................................................................................................................................................................................................................................................... 4

How does it work? ................................................................................................................................................................................................................................................................................................................. 6

Setting the FC switches for NPIV ............................................................................................................................................................................................................................................................................ 8

Best practice, zoning, and multi-pathing considerations for FC and FCoE ....................................................................................................................................................................... 9

Firmware upgrades with Persistent Ports .................................................................................................................................................................................................................................................... 10

Using Matched Set VLUN templates with Persistent Ports ......................................................................................................................................................................................................... 10

HPE Virtual Connect considerations with Persistent Ports.......................................................................................................................................................................................................... 11

Technical white paper

Technical white paper Page 2

Terminology • N_Port ID Virtualization (NPIV): A Fibre Channel (FC) facility that allows multiple N_Port IDs to share a single physical N-Port on an

FC switch.

• Host facing port: A port on a host bus adapter (HBA) in an HPE 3PAR StoreServ array that has been configured to provide host access to the array.

• Partner nodes: A pair of nodes on an HPE 3PAR StoreServ array that are connected to a common set of disk enclosures (chassis).

• Host Port Native Identity: The primary port identity of an HPE 3PAR host facing port (for example, 0:1:1).

• Host Port Guest Identity: A host port’s “Guest” identity is associated with a corresponding host facing port’s “Native” identity on its partner node (For example, on Native port 0:1:1 the Guest identity would be 1:1:1.). This corresponding Guest identity on the partner node will be the same slot and port as the partner node’s Native identity. The host port “Guest” identity is used to log into the SAN fabric by a node’s partner if the node owning the Native identity becomes unavailable.

• FCoE: FCoE is a computer network technology that encapsulates FC frames onto lossless 10 or 100 Gb Ethernet networks.

• iSCSI: iSCSI is an IP-based protocol for linking data storage devices over an IP network and transferring data by carrying SCSI commands over IP networks.

Executive summary Mission-critical tier-1 storage environments require extreme high availability (HA). Tier-1 customers running hundreds (or thousands) of servers in an enterprise environment feel that a dependency on host multipathing failover software during firmware upgrades, node failures, or in response to a “loss_sync” event (a physical layer problem between the storage array and the switch) introduces the risk of a service disruption and hence should be avoided. HPE 3PAR Persistent Ports technology allows for a non-disruptive environment (from the host multi-pathing point of view) where host-based multi-pathing software will not be involved in maintaining server connectivity to the storage during firmware upgrades, in the event of a node failure, in response to an array port being taken offline administratively, or as the result of a hardware failure in the SAN fabric that results in the storage array losing physical connectivity to the fabric.

Persistent Ports technology does not negate the need for properly installed, configured, and maintained host multi-pathing software. Although Persistent Ports technology will isolate a server from the need for path failover during firmware upgrades, in the event of a fabric hardware failure resulting in a “loss_sync” or in the event a node becomes available due to a panic or loss of power, it will not protect from cable problems or host HBA failures that do not result in a “loss_sync” on the storage array node. A properly configured multi-pathing environment provides protection from these events.

Features and benefits HPE 3PAR Persistent Ports functionality works for the following transport layers:

• FC

• FCoE

• iSCSI

HPE 3PAR Persistent Ports functionality provides transparent and uninterrupted failover in response to the following events:

• HPE 3PAR OS firmware upgrade

• Node maintenance that requires the node to be taken offline (e.g., adding a new HBA)

• HPE 3PAR node failure

• HPE 3PAR array “loss_sync” to the FC fabric

• HPE 3PAR array “loss_sync” to an iSCSI network

• Array host ports being taken offline administratively

In most cases, no configuration changes are required on the servers in an environment using the HPE 3PAR StoreServ system for Persistent Ports functionality to be leveraged. All configuration work is done on the HPE 3PAR StoreServ and SAN fabric.

https://www.hpe.com/us/en/storage/3par.html



What’s new with Persistent Ports on HPE 3PAR? Persistent Ports support in HPE 3PAR OS 3.1.2 was limited to FC connectivity only, and it could only respond to a node being taken offline for firmware upgrades, node failure, or HPE 3PAR OS CLI controlport command. Starting with HPE 3PAR OS 3.1.3 Persistent Ports was extended to include FCoE and iSCSI connections but loss sync was not supported for these connections.

Starting with HPE 3PAR OS 3.2.2, the ability to detect and respond to a “loss_sync” event has been extended to iSCSI connections. The ability to detect “loss_sync” is still not supported for FCoE connections.

Note Starting with HPE 3PAR OS 3.2.2, Persistent Ports has the ability to detect and recover from a “loss_sync” event for an iSCSI connection to the array.

Note Even with HPE 3PAR OS 3.2.2 Persistent Ports does not recover from “loss_sync” events for FCoE connections to the array.

What does it do? With HPE 3PAR Persistent Ports technology, each host facing port (FC, FCoE, and iSCSI) on an HPE 3PAR StoreServ node has both a “Native” and a “Guest” identity associated with it. The “Guest” identity on a node corresponds to a “Native” port on the partner HPE 3PAR node in a node pair. To provide greater resiliency and to avoid dependency on host multi-pathing software, HPE 3PAR Persistent Ports technology transparently configures the network to redirect I/O requests from the “Native” port on one node to the “Guest” port on another node in response to firmware upgrades, node failure, “loss_sync” events, or administrative requests. This redirection is done in a manner that is transparent to high-level host software sending I/O requests to the node being upgraded and hence removes the need for host-level multi-pathing software to respond to the event.

It is possible that in some cases, certain OSs may log errors as a result of loss_sync or firmware upgrades that results from perturbations on the SAN fabric, but these perturbations do not result in the multi-pathing software having to intervene resulting in a potential interruption to high-level host I/Os.

For FC and FCoE connections, Persistent Ports technology leverages SAN fabric NPIV functionality for transparent migration of a server’s “Native” port Worldwide Name (WWN) to a “Guest” port on the other node in the node pair. For iSCSI connections, it migrates a port’s IP address from the Native port on one node and the Guest port on the other node in a node pair (This IP address migration is followed by an unsolicited Address Resolution Protocol [ARP] to force the updating of IP to MAC mappings on the network).

Requirements Persistent Ports functionality supports HPE 3PAR OS 3.1.2 and later. Starting with HPE 3PAR OS 3.1.3, support for FCoE-connected hosts and iSCSI-connected hosts is added with the ability to detect an array node suffering “loss_sync” (a physical layer problem occurring between the HPE 3PAR node and the switch it is connected to) for Fibre Channel connected hosts but not supported for either FCoE connections or iSCSI connections. Starting with HPE 3PAR OS 3.2.2, the ability to detect and recover from “loss_sync” resulting from a cable pull for iSCSI-connected hosts is added. There is still no ability to detect and recover from a “loss_sync” for FCoE connected hosts. There is no Persistent Ports support on pre-HPE 3PAR OS 3.1.2 releases.

Note HPE 3PAR Persistent Ports functionality does not detect and recover from a “loss_sync” event for FCoE connected hosts.

For HPE 3PAR StoreServ FC host ports, the following requirements must be met:

• The same host port on host facing HBAs in the nodes in a node pair must be connected to the same FC fabric and preferably different FC switches on the fabric (for example, 0:1:1 and 1:1:1).

• The host facing HBAs must be set to “target” mode.

• The host facing HBAs must be configured for point-to-point connection (no support for “loop”).

• The FC fabric being used must support NPIV and have NPIV enabled.



For HPE 3PAR StoreServ FCoE host ports, the following requirements must be met:

• The same converged network adapter (CNA) port on host facing HBAs in the nodes in a node pair must be connected to the same FCoE network and preferably different FCoE switches on the network (for example, 0:1:1 and 1:1:1).

• The host facing CNAs must be set to “target” mode.

• The FCoE network being used must support NPIV and have NPIV enabled.

For HPE 3PAR StoreServ iSCSI host ports, the following requirements must be met:

• The same host port on host facing CNAs in the nodes in a node pair must be connected to the same IP network and preferably different IP switches on the fabric (for example, 0:1:1 and 1:1:1).

There are no special requirements placed on the server HBAs and CNAs other than that they must be supported as per Single Point of Connectivity Knowledge (SPOCK). See hpe.com/storage/spock.

Configuration considerations Persistent Ports requires that corresponding “Native” and “Guest” host ports on a node pair be connected to the same FC fabric or IP network, and the switches they are connected to must support and be configured for NPIV in the case of FC and FCoE. This means that for a minimum configuration to provide Persistent Ports functionality, where the node pair is connected to redundant FC SAN fabrics, each node in a node pair must have at least two FC host ports cabled with one port connected to each fabric. Figure 1 shows a minimum two-fabric redundant configuration that supports Persistent Ports for FC and Figure 2 shows a configuration that will not support Persistent Ports functionality. Note that the same slot:port on each node in the node pair are connected to the same FC fabric.

Figure 1. Minimum configuration—both the “Native” and “Guest” ports must be connected to the same FC fabric, FCoE network, or iSCSI network

In minimum configurations, HPE 3PAR StoreServ 7000 arrays that only have default FC port configurations (two ports) may not have enough host ports to enable Persistent Ports in a dual fabric (Figure 2) if ports are used for HPE 3PAR Remote Copy software or HPE 3PAR Peer Motion software. For these configurations, the optional four port FC HBA is required to provide enough FC host ports per node to allow Persistent Ports.

http://www.hpe.com/storage/spock


Figure 2. Minimum configuration—both the “Native” and “Guest” ports must be connected to the same FC fabric, FCoE network, or iSCSI network

A best practice configuration will utilize a network mesh architecture where the “Native” and “Guest” ports in a Persistent Ports pair are connected to different switches in the mesh (This is the case whether the connections are FC, FCoE, or iSCSI.). This allows the “loss_sync” functionality of Persistent Ports to recover from a switch failure in an FC-based configuration. See Figure 3 for a best practice Persistent Ports configuration using FC that will protect from a node “loss_sync” event in the event of a switch failure.

Figure 3. Example of a best practice Persistent Ports configuration showing an FC mesh


How does it work? With Persistent Ports, FC, FCoE, or iSCSI host ports on an HPE 3PAR StoreServ node pair have both a “Native” and a “Guest” identity associated with them. By having a Native and a Guest identity associated with its host ports, a node can be the backup for its partner node should that node fail or lose connectivity to its network. A host port on a node can only have a Native and Guest identity if both of it and its partner node’s corresponding host port (the same port on an HBA or CAN in the same slot) are connected to a common SAN fabric, FCoE network, or iSCSI subnet.

For FC and FCoE, the Native identity for a port is the port WWN that port has on a given node. For example, 20110002AC0008EB for the HBA in Node 0, Slot 1, Port 2. Node 0’s partner node, Node 1, has a Native port with a similar identity, 21110002AC0008EB, for the HBA in Node 1, Slot 1, Port 2 (see Figure 4). So, for this example the “Native” and “Guest” identities for nodes 0 and 1 would be:

Node 0—20110002AC0008EB (Native) and 21110002AC0008EB (Guest)

Node 1—21110002AC0008EB (Native) and 20110002AC0008EB (Guest)

Figure 4. Native and Guest identity on host ports

It’s a little different with iSCSI since iSCSI does not use WWNs. In an iSCSI configuration, the “Native” and “Guest” port WWNs do not move; the IP address configured on the Native port moves from the Native port to the Guest port, and an unsolicited ARP is transmitted to notify the network for the change in the IP to MAC address mapping (see Figure 5).


Figure 5. Native and Guest IP identity on CNA ports

With Persistent Ports, during a firmware upgrade, a node failure, if a node fails, or in the case of FC loses connectivity to a fabric due to a “loss_sync”, the “Native” identity of a port on one node in a node pair becomes the “Guest” identity on the corresponding port on that node’s partner in response to a failure or CLI command. See Figure 6 for an example of Persistent Ports failover as the result of a node failure.

Figure 6. If a node fails its partner node’s port, “Guest” identity takes over

In the event of a failure or a planned activity on Node 0 in Figure 6, the Guest port on Node 1 logs on to the fabric using the Guest identity for port 0:1:1. This happens in a few seconds and is not visible at the SCSI layer.

When a Persistent Port failover occurs, the FailoverState of the host ports can be one of the following states:

• none: Failover not in operation.

• failover_pending: Failover to partner request has been issued but not yet completed (transient state).


• failed_over: This port is failed over to its partner.

• failback_pending: Failback request has been issued but not yet completed (transient state).

• active: The partner port is failed over to this port.

• active_down: The partner port failed over to this port, but this port is down, e.g., cable missing.

• active_failed: The partner port failed over to this port, but the action failed, e.g., FC switch did not have NPIV enabled.

For example, with 0:1:1 failing over to 1:1:1, port 0:1:1 has the FailoverState of failover_pending:

Test_sys cli% showport 0:1:1 1:1:1

N:S:P Mode State ----Node_WWN---- -Port_WWN/HW_Addr- Type Protocol Label Partner FailoverState

0:1:1 target ready 2FF70002AC0008EB 20110002AC0008EB host FC - 1:1:1 failover_pending

1:1:1 target ready 2FF70002AC0008EB 21110002AC0008EB host FC - 0:1:1 active

Once the failover is complete, observe that 0:1:1 has the FailoverState as failed_over and 1:1:1 has the FailoverState of active:



0:1:1 target ready 2FF70002AC0008EB 20110002AC0008EB host FC - 1:1:1 failed_over

1:1:1 target ready 2FF70002AC0008EB 21110002AC0008EB host FC - 0:1:1 active

If the path fails on 1:1:1, observe that 1:1:1 has the FailoverState as active_down (This is the case where host port 1:1:1 on Node 1 is active for both the Native and Guest identities but fails.):



0:1:1 target ready 2FF70002AC0008EB 20110002AC0008EB host FC - 1:1:1 failed_over

1:1:1 target ready 2FF70002AC0008EB 21110002AC0008EB host FC - 0:1:1 active_down

Setting the FC switches for NPIV The FC fabric and the HPE 3PAR StoreServ node pair is connected to support NPIV and be configured for NPIV.

On Brocade, use the portCfgNPIVPort command to set NPIV on the SAN switch the HPE 3PAR StoreServ is connected to:

portCfgNPIVPort

Enables or disables NPIV functionality on a port and sets the per-port login limit.

Synopsis portcfgnpivport --enable [slot/]port

portcfgnpivport --disable [slot/]port

portcfgnpivport --setloginlimit [slot/]port login_limit

On Cisco, use the following steps to enable or disable NPIV on the switch when the HPE 3PAR StoreServ is connected:

Command Purpose

switch# config t Enters configuration mode

switch(config)# feature npiv Enables NPIV for all VSANs on the switch

switch(config)# no feature npiv Disables (default) NPIV on the switch


Best practice, zoning, and multi-pathing considerations for FC and FCoE Following the currently published best practices for zoning HPE 3PAR arrays will enable Persistent Ports functionality. Be aware that some previous best practices did not require server HBAs to have a physical path to either nodes or the same slot:port in a node pair if it did have multiple paths. Also, previous best practices only required that a server HBA be connected to a single HPE 3PAR StoreServ node. As we have seen, for Persistent Ports to work, a server HBA must have a physical path (be cabled) to both the “Native” port and a path to the “Guest” port on a node pair (see Figure 1), which means it must be cabled to both nodes in the node pair.

When setting up zoning, it is not necessary for the HBA port to be zoned to both the Native and Guest port on the array if you are using WWN zoning because Persistent Ports will move one port’s WWN to the port on the other node in the event of a failure. If you are using port-based zoning, the Native identity from both ports in the Persistent Ports pair must be zoned to the server HBA; however, they may be in the same or different zone. If you do zone the server HBA to both ports in a Persistent Ports pair, then load-balancing software can be used to load balance host I/Os across the Native identity ports on the two nodes. See Figure 7 for an example of zoning to both nodes for Persistent Ports support that will allow load balancing across the nodes. See Figure 8 for an example of zoning to only one node, which is supported, but will not allow a single server’s HBA to I/O load balance across the two Native identity ports on a node pair.

Once again, the server HBA can use load-balancing algorithms across the nodes if it is zoned to both Native identity ports in a Persistent Ports pair.

Figure 7. Host HBA zoning example for Persistent Ports support that can support load balancing across both pairs of Native identity ports (port pair 0:1:1 and 1:1:1 and port pair 0:1:2 and 1:1:2)


Figure 8. Host HBA zoning example for Persistent Ports support that cannot support load balancing across the Native identity ports (port pair 0:1:1 and 1:1:1 and port pair 0:1:2 and 1:1:2)

If an existing HPE 3PAR StoreServ is being upgraded to HPE 3PAR OS 3.1.3 or later and Persistent Ports functionality is desired, check the cabling of the existing servers connected to the array and help ensure that common HBA slot:port pairs on partner nodes are connected to the same fabric. If they aren’t, then zoning changes and or cabling changes on the SAN fabric will be required.

Warning SAN cabling changes and SAN zoning changes may require configuration changes on the server to help ensure the server’s ability to access its logical unit numbers (LUNs) on the storage.

Firmware upgrades with Persistent Ports Firmware upgrade on the HPE 3PAR StoreServ involves rebooting one controller node at a time. With Persistent Ports, when a node reboots as part of the upgrade process, its partner node logs in to the FC fabric using the “Guest” identity. This all occurs in a manner that is transparent to the host using the HPE StoreServ array, so no multi-pathing failover has to occur as a result. When the node being upgraded comes back online, it logs in to the fabric using its original Native port identity and all the servers’ I/O requests are then transparently redirected back to that node.

Using Matched Set VLUN templates with Persistent Ports If you are using “Matched Set” VLUN templates to export VVs, the VLUNs are only exported on the physical array ports that have exports associated to them. This restriction can interfere with the mechanism of Persistent Ports. When using Matched Set VLUN templates for exports and Persistent Ports together, you must enable the “FailoverMatchedSet” system configuration parameter using the CLI. Execute the following CLI command to enable this parameter:

# setsys FailoverMatchedSet yes

The default for this setting is disabled or “no.”


HPE Virtual Connect considerations with Persistent Ports The HPE Virtual Connect (VC) technology simplifies the networking configuration for the server administrator using an HPE BladeSystem c-Class environment. The HPE VC technology virtualizes the connections between the server and the LAN and SAN network infrastructure.

VC firmware 3.70 or later supports two VC SAN configurations, Fabric Attach and Direct Attach. With Direct Attach FC support, users can now connect HPE 3PAR Storage Array systems directly to the HPE VC FlexFabric module with no need for an expensive intermediate SAN fabric. In addition to being much more cost-efficient, management of your storage solution is made easier. Valuable IT resources are freed up, along with reduced costs.

The VC Direct Attach auto-configured zoning (aka Implicit Zoning) allows servers connected to a Direct-Attach Fabric access to storage devices connected to the uplinks within that Direct-Attach Fabric. There is no manual zoning that is configuration required. The zoning is automatically configured based on the VC fabric definitions and profile SAN connectivity selections.

When a VC Direct-Attach Fabric is using multiple uplinks (Figure 6), hosts will see as many paths to the storage as there are uplinks to the HPE 3PAR StoreServ array. The concept of login balancing or login redistribution is not applicable in this case as these concepts are only provided on uplinks within a VC Fabric-Attach Fabric.

Persistent Ports requires that corresponding “Native” and “Guest” host ports on a node pair be connected to the same FC fabric. This means that for a minimum configuration to provide Persistent Ports functionality where the node pair is connected to redundant VC SAN fabrics, each node in a node pair must have at least two FC host ports configured with one port connected to each fabric. Figure 9 shows configuration that supports Persistent Ports configuration functionality.

Uplinks from a VC module in one bay should be connected to the same port pair (Native and Guest) in a pair nodes’ node.

Figure 9. StoreServ host port connections to HPE VC FlexFabric modules

A maximum of four uplinks per VC are available, and as a result, a total of eight uplinks from both VCs in a blade chassis are available to be connected to the array. The host will see as many paths as the number of the uplinks from the VC connect to the HPE 3PAR StoreServ array.

The user can create multiple VC SANs to isolate servers and in this case isolate the server’s workload (Figure 10).




Figure 10. Using multiple VC SANs to isolate servers

When connecting more than one HPE 3PAR StoreServ array, it is recommended to create multiple VC SANs to isolate the traffic of the arrays.

Figure 11. Using multiple VC SANs to isolate multiple arrays sharing a pair of HPE VC FlexFabric modules

Technical white paper

Share now

Get updates

© Copyright 2012–2015, 2019 Hewlett Packard Enterprise Development LP. The information contained herein is subject to change without notice. The only warranties for Hewlett Packard Enterprise products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. Hewlett Packard Enterprise shall not be liable for technical or editorial errors or omissions contained herein.

4AA4-4545ENW, January 2019, Rev. 6

When connecting multiple HPE 3PAR StoreServ arrays, the user should consider whether a single pair of VC FlexFabric modules is sufficient or whether they should consider adding multiple VC FlexFabric module pairs. Please note that the maximum number of HPE 3PAR StoreServ arrays supported by a VC FlexFabric in a Direct Attach mode is four HPE 3PAR StoreServ arrays.

Per the HPE VC user and installation guide, it is recommended that all VC domains connected to the same HPE 3PAR Storage system use different HPE predefined ranges of WWN addresses. This will help avoid storage networking issues and potential loss of data associated with duplicate WWNs on the HPE 3PAR StoreServ array system.

Learn more at hpe.com/us/en/storage/3par.html

https://www.hpe.com/global/hpechat/index.html?jumpid=Collaterals_4AA4-4545ENW

http://www.hpe.com/info/getupdated


Documents

HPE 3PAR StoreServ Persistent Ports technical white paper › V2 › GetPDF.aspx › 4AA4... · Terminology • N_Port ID Virtualization (NPIV): A Fibre Channel (FC) facility that