Netqueues_1_3_2

Embed Size (px)

Citation preview

  • 8/6/2019 Netqueues_1_3_2

    1/20

    NetQueue API 1.3

    1VMware Confidential

    ESX Server

    NetQueue Programming APIVersion 1.3

    10/15/2007

    VMware Confidential

  • 8/6/2019 Netqueues_1_3_2

    2/20

    NetQueue API 1.3

    2VMware Confidential

    Revision History for NetQueue API (version 1.3)

    Revision Date Name Description1.0 1.2 Gagan Arneja Initial draft of Netchannels API1.3 06/11/2007 Gagan Arneja,

    Caixue LinChanged Netchannels to NetQueue

    10/15/2007 Gagan Arneja,Caixue Lin

    Deprecated VLAN only filter;Added a FAQ list as Appendix

  • 8/6/2019 Netqueues_1_3_2

    3/20

    NetQueue API 1.3

    3VMware Confidential

    Table of Contents

    Introduction......................................................................................................................................... 4NetQueue API registration.................................................................................................................. 4NetQueue API commands................................................................................................................... 5Miscellaneous ................................................................................................................................... 18Appendix----FAQ ............................................................................................................................. 19

  • 8/6/2019 Netqueues_1_3_2

    4/20

    NetQueue API 1.3

    4VMware Confidential

    Introduction

    NetQueue API provides a standard interface for programming Network InterfaceCards (NICs) with multiple receive and transmit queues. The API consists ofmeans to discover receive and transmit queues and program filters on thesequeues. Note that this version of NetQueue API currently implements only thesupport for receive queues. NetQueue API is an extension of the VMklinux API.

    NetQueue API registration

    A VMklinux NIC driver calls VMKNETDDI_REGISTER_QUEUEOPS(ndev, ops) to register aNetQueue callback entry point function. The callback function is invoked by theVMkernel to request setting up and tearing down receive filters on the NIC.

    VMKNETDDI_REGISTER_QUEUEOPS is internally defined as:

    #define VMKNETDDI_REGISTER_QUEUEOPS(ndev, ops) \ndev->netqueue_ops = ops;

    Arguments: IN: netdev - Pointer to the VMklinux struct net_devicefor the device

    IN: ops - Pointer to the NetQueue API entrypoint function

    VMKNETDDI_REGISTER_QUEUEOPS() must be called for each VMklinux net_device supported by the driver. The call must be invoked prior to the VMklinuxregister_netdev call.

    NetQueue API entry point

    NetQueue API entry point is defined as:

    typedef int (*vmknetddi_queueops_f)(vmknetddi_queueops_op_t cmd, void *args);

    Arguments: IN: cmd - NetQueue API commandIN: args - Opaque pointer to command arguments

    Return values: VMKNETDDI_QUEUEOPS_OK - SuccessVMKNETDDI_QUEUEOPS_ERR - Failure

    The cmd argument specifies the command to be performed by the driver. Variouscommands and their arguments are described in the next section.

  • 8/6/2019 Netqueues_1_3_2

    5/20

    NetQueue API 1.3

    5VMware Confidential

    NetQueue API commands

    The following NetQueue commands are supported by the current version of the API:

    VMKNETDDI_QUEUEOPS_OP_GET_VERSION

    Description: VMKNETDDI_QUEUEOPS_OP_GET_VERSION returns the version ofNetQueue API supported by the driver.

    Argument: typedef struct _vmknetddi_queueop_get_version_args_t {OUT: u16 major;OUT: u16 minor;

    } vmknetddi_queueop_get_version_args_t ;

    major : Major versionminor : Minor version

    Return values: Success - VMKNETDDI_QUEUEOPS_OK Failure VMKNETDDI_QUEUEOPS_ERR

    Notes: The driver is supposed to fill in major and minor with theversion of NetQueue API it was compiled against. For allpurposes, the driver should call the predefinedvmknetddi_queueops_version(args) function, which will fill inthe right values.

    Example: mydriver_netqueue_ops(vmknetddi_queueops_op_t op, void *args){

    switch (op) {...case VMKNETDDI_QUEUEOPS_OP_GET_VERSION:

    return vmknetddi_queueops_version(args);break;

    ....}

    }

  • 8/6/2019 Netqueues_1_3_2

    6/20

    NetQueue API 1.3

    6VMware Confidential

    VMKNETDDI_QUEUEOPS_OP_GET_FEATURES

    Description: VMKNETDDI_QUEUEOPS_OP_GET_FEATURES returns NetQueuecapabilities supported by the NIC. The following capabilitiesare defined in the current version of the API:

    VMKNETDDI_QUEUEOPS_FEATURE_NONE - No Netqueue capabilitiesVMKNETDDI_QUEUEOPS_FEATURE_RXQUEUES The NIC has multiple

    receive queuesVMKNETDDI_QUEUEOPS_FEATURE_TXQUEUES The NIC has multiple

    transmit queues. (Note: Thisfeature is not used in thecurrent version of the API)

    Argument: typedef struct _vmknetddi_queueop_get_features_args_t {IN: struct net_device *netdev; OUT: vmknetddi_queueops_features_t features;

    } vmknetddi_queueop _get_features_args_t;

    netdev: Pointer to struct net_devicevmknetddi_queueops_features_t : Features supported. It can be a

    logical OR of the following:

    VMKNETDDI_QUEUEOPS_FEATURE_RXQUEUES ,VMKNETDDI_QUEUEOPS_FEATURE_TXQUEUES

    Return values: Success - VMKNETDDI_QUEUEOPS_OK Failure VMKNETDDI_QUEUEOPS_ERR

    Notes: The driver is supposed to fill in features with theNetQueue capabilities it supports. features is a bitmaskthat can be a logical OR of the above definedVMKNETDDI_QUEUEOPS_FEATURE_* values.

    Example: mydriver_netqueue_ops(vmknetddi_queueops_op_t op, void *args){

    switch (op) {...case VMKNETDDI_QUEUEOPS_OP_GET_FEATURES:

    my_args = (vmknetddi_queueop_get_features_args_t *)args;

    my_args->features = VMKNETDDI_QUEUEOPS_FEATURE_NONE;my_args->features |=

    VMKNETDDI_QUEUEOPS_FEATURE_RXQUEUES;return VMKNETDDI_QUEUEOPS_OK;

    ....}

    }

  • 8/6/2019 Netqueues_1_3_2

    7/20

    NetQueue API 1.3

    7VMware Confidential

    VMKNETDDI_QUEUEOPS_OP_GET_QUEUE_COUNT

    Description: VMKNETDDI_QUEUEOPS_OP_GET_QUEUE_COUNT returns the number ofqueues supported by the NIC.

    Argument: typedef struct _vmknetddi_queueop_get_queue_count_args_t {IN : struct net_device *netdev;IN : vmknetddi_queueops_queue_t type;OUT: u16 count;

    } vmknetddi_queueop_get_queue_count_args_t;

    netdev: Pointer to struct net_devicetype: type specifies the type of queue to query. It can be

    either of the following:

    VMKNETDDI_QUEUEOPS_QUEUE_TYPE_RX ,VMKNETDDI_QUEUEOPS_QUEUE_TYPE_TX

    Currently, only VMKNETDDI_QUEUEOPS_QUEUE_TYPE_RX issupported. Support will be added for additional

    queue types in future versions of the API.

    count : Number of queues supported.

    Return values: Success - VMKNETDDI_QUEUEOPS_OK Failure VMKNETDDI_QUEUEOPS_ERR

    Notes: The driver is supposed to fill in count with thecount of queues of the specified type it supports.

    Example: mydriver_netqueue_ops(vmknetddi_queueops_op_t op, void *args){

    switch (op) {...case VMKNETDDI_QUEUEOPS_OP_GET_QUEUE_COUNT:my_args =

    (vmknetddi_queueop_get_queue_count_args_t*)args;

    if (my_args->type == VMKNETDDI_QUEUEOPS_QUEUE_TYPE_RX) {my_args->count = netdev->n_rx_queues;return VMKNETDDI_QUEUEOPS_OK;

    }else {

    return VMKNETDDI_QUEUEOPS_ERR;}....

    }

    }

  • 8/6/2019 Netqueues_1_3_2

    8/20

    NetQueue API 1.3

    8VMware Confidential

    VMKNETDDI_QUEUEOPS_OP_GET_FILTER_COUNT

    Description: VMKNETDDI_QUEUEOPS_OP_GET_FILTER_COUNT returns the number offilters supported by each receive queue on the NIC.

    Argument: typedef struct _vmknetddi_queueop_get_filter_count_args_t {IN: struct net_device *netdev;IN: vmknetddi_queueops_queue_t type;OUT: u16 count;

    } vmknetddi_queueop_get_filter_count_args_t;

    netdev: Pointer to struct net_device

    type: type specifies the type of queue to query. It can beeither of the following:

    VMKNETDDI_QUEUEOPS_QUEUE_TYPE_RX ,VMKNETDDI_QUEUEOPS_QUEUE_TYPE_TX

    VMKNETDDI_QUEUEOPS_QUEUE_TYPE_RX is the only valid

    option in this version of the API.

    count : Number of filters supported on each queue

    Return values: Success - VMKNETDDI_QUEUEOPS_OK Failure VMKNETDDI_QUEUEOPS_ERR

    Notes: The driver is supposed to fill in count with thenumber of filters it supports on a receive queue.

    Example: mydriver_netqueue_ops(vmknetddi_queueops_op_t op, void *args){

    switch (op) {...case VMKNETDDI_QUEUEOPS_OP_GET_FILTER_COUNT:my_args =

    (vmknetddi_queueop_get_filter_count_args_t *)args;

    if (my_args->type == VMKNETDDI_QUEUEOPS_QUEUE_TYPE_RX) {my_args->count = netdev->n_rx_queue_filters;return VMKNETDDI_QUEUEOPS_OK;

    }else {

    return VMKNETDDI_QUEUEOPS_ERR;}....

    }

    }

  • 8/6/2019 Netqueues_1_3_2

    9/20

    NetQueue API 1.3

    9VMware Confidential

    VMKNETDDI_QUEUEOPS_OP_ALLOC_QUEUE

    Description: VMKNETDDI_QUEUEOPS_OP_ALLOC_QUEUE allocates an availablequeue.

    Argument: typedef struct _vmknetddi_queueop_alloc_queue_args_t {IN : struct net_device *netdev;IN : vmknetddi_queueops_queue_t type;OUT: vmknetddi_queueops_queueid_t queueid;

    } vmknetddi_queueop_alloc_queue_args_t;

    netdev: Pointer to struct net_device

    type: type specifies the type of queue to alloc. OnlyVMKNETDDI_QUEUEOPS_QUEUE_TYPE_RX is supported in thisversion of the API

    queueid : is the identifier of the allocated queue.The driver should choose a unique 16bit valueto represent each queue internally. This

    queue identifier must then be wrapped into avalue that can be used by VMkernel by callingVMKNETDDI_QUEUEOPS_MK_RX_QUEUEID() orVMKNETDDI_QUEUEOPS_MK_TX_QUEUEID().

    Return values: Success - VMKNETDDI_QUEUEOPS_OK Failure VMKNETDDI_QUEUEOPS_ERR

    Notes: The driver is supposed to fill in queueid with thequeue id of the allocated queue. The driver must useVMKNETDDI_QUEUEOPS_MK_RX_QUEUEID(id) to convert its internalqueue id value to a format that can be used by the VMkernel.

    Example: mydriver_netqueue_ops(vmknetddi_queueops_op_t op, void *args){

    switch (op) {...case VMKNETDDI_QUEUEOPS_OP_ALLOC_QUEUE:

    my_args = (vmknetddi_queueop_alloc_queue_args_t*)args;

    if (my_args->type == VMKNETDDI_QUEUEOPS_QUEUE_TYPE_RX) {

    my_args->queueid = VMKNETDDI_QUEUEOPS_MK_RX_QUEUEID(netdev-

    >next_avail);

    netdev->queue_bitmap[next_avail] = ALLOCATED;netdev->next_avail++;return VMKNETDDI_QUEUEOPS_OK;

    }else {

    return VMKNETDDI_QUEUEOPS_ERR;}

    ....}

    }

  • 8/6/2019 Netqueues_1_3_2

    10/20

    NetQueue API 1.3

    10VMware Confidential

    VMKNETDDI_QUEUEOPS_OP_FREE_QUEUE

    Description: VMKNETDDI_QUEUEOPS_OP_FREE_QUEUE frees a previously allocatedqueue

    Argument: typedef struct _vmknetddi_queueop_free_queue_args_t {IN : struct net_device *netdev;IN : vmknetddi_queueops_queueid_t queueid;

    } vmknetddi_queueop_free_queue_args_t;

    netdev: Pointer to struct net_device

    queueid : queue identifier of the allocated queue.The driver must useVMKNETDDI_QUEUEOPS_QUEUEID_VAL(queueid) to extractits internal queue id value from the one passedin.

    Return values: Success - VMKNETDDI_QUEUEOPS_OK Failure VMKNETDDI_QUEUEOPS_ERR

    Example: mydriver_netqueue_ops(vmknetddi_queueops_op_t op, void *args){

    switch (op) {...case VMKNETDDI_QUEUEOPS_OP_FREE_QUEUE:

    my_args = (vmknetddi_queueop_free_queue_args_t *) args;

    my_qid = VMKNETDDI_QUEUEOPS_QUEUEID_VAL(my_args->queueid);

    netdev->queue_bitmap[my_qid] = FREE;return VMKNETDDI_QUEUEOPS_OK;

    ....}

    }

  • 8/6/2019 Netqueues_1_3_2

    11/20

    NetQueue API 1.3

    11VMware Confidential

    VMKNETDDI_QUEUEOPS_OP_GET_QUEUE_VECTOR

    Description: VMKNETDDI_QUEUEOPS_OP_GET_QUEUE_VECTOR returns the interruptvector assigned to the queue

    Argument: typedef struct _vmknetddi_queueop_get_queue_vector_args_t {IN: struct net_device *netdev;IN: vmknetddi_queueops_queueid_t queueid;OUT: u16 vector;

    } vmknetddi_queueop_get_queue_vector_args_t;

    netdev: Pointer to struct net_device

    queueid : queue identifier of the allocated queue.The driver must useVMKNETDDI_QUEUEOPS_QUEUEID_VAL(queueid) to extractits internal queue id value from the one passedin.

    vector : Interrupt vector assigned to the queue

    Return values: Success - VMKNETDDI_QUEUEOPS_OK Failure VMKNETDDI_QUEUEOPS_ERR

    Notes: The driver is supposed to fill in vector with theinterrupt vector being used for queueid . A NICsupporting MSI-X can request a block of vectors from thekernel. Ideally, these allocated vectors are divvied upamongst different queues. The driver should fill in vector with the vector being used for the asked queue.

    Example: mydriver_netqueue_ops(vmknetddi_queueops_op_t op, void *args){

    switch (op) {...case VMKNETDDI_QUEUEOPS_OP_GET_QUEUE_VECTOR:

    my_args = (vmknetddi_queueop_get_queue_vector_args_t *)

    args;my_qid =

    VMKNETDDI_QUEUEOPS_QUEUEID_VAL(my_args->queueid);

    my_args->vector = my_driver_assigned_vector(my_qid);return VMKNETDDI_QUEUEOPS_OK;

    ....}

    }

  • 8/6/2019 Netqueues_1_3_2

    12/20

    NetQueue API 1.3

    12VMware Confidential

    VMKNETDDI_QUEUEOPS_OP_GET_DEFAULT_QUEUE

    Description: VMKNETDDI_QUEUEOPS_OP_GET_DEFAULT_QUEUE queries the defaultqueue used for packets not matching any filter criteria

    Argument: typedef struct _vmknetddi_queueop_get_default_queue_args_t {IN: struct net_device *netdev;IN: vmknetddi_queueops_queue_t type;OUT: vmknetddi_queueops_queueid_t queueid;

    } vmknetddi_queueop_get_default_queue_args_t;

    netdev: Pointer to struct net_device

    type: type specifies the type of queue. OnlyVMKNETDDI_QUEUEOPS_QUEUE_TYPE_RX is supported in thisversion of the API

    queueid : queue id of the default queue

    Return values: Success - VMKNETDDI_QUEUEOPS_OK

    Failure VMKNETDDI_QUEUEOPS_ERR

    Notes: The driver is supposed to fill in queueid with the queueid of the default queue. The driver must useVMKNETDDI_QUEUEOPS_MK_RX_QUEUEID(id) to convert its internalqueue id value to a format that can be used by the VMkernel.

    Example: mydriver_netqueue_ops(vmknetddi_queueops_op_t op, void *args){

    switch (op) {...case VMKNETDDI_QUEUEOPS_OP_GET_DEFAULT_QUEUE:

    my_args = (vmknetddi_queueop_get_default_queue_args_t *)args;

    if (my_args->type == VMKNETDDI_QUEUEOPS_QUEUE_TYPE_RX) {

    my_args->queueid = VMKNETDDI_QUEUEOPS_QUEUEID_VAL(0);

    return VMKNETDDI_QUEUEOPS_OK;}else {

    return VMKNETDDI_QUEUEOPS_ERR;}

    ....}

    }

  • 8/6/2019 Netqueues_1_3_2

    13/20

    NetQueue API 1.3

    13VMware Confidential

    VMKNETDDI_QUEUEOPS_OP_APPLY_RX_FILTER

    Description: VMKNETDDI_QUEUEOPS_OP_APPLY_RX_FILTER programs a receivefilter on a previously allocated queue

    Argument: typedef struct _vmknetddi_queueop_apply_rx_filter_args_t {IN: struct net_device *netdev;IN: vmknetddi_queueops_queueid_t queueid;IN: vmknetddi_queueops_filter_t filter;OUT: vmknetddi_queueops_filterid_t filterid;

    } vmknetddi_queueop_apply_rx_filter_args_t;

    netdev: Pointer to struct net_device

    queueid : queue id of the queue to apply receive filteron

    filter: receive filter to apply

    filterid : is the identifier of the allocated filter.

    The driver should choose a unique 16bit valueto represent each filter internally. Thisfilter identifier must then be wrapped into avalue that can be used by VMkernel by callingVMKNETDDI_QUEUEOPS_MK_FILTERID()

    Return values: Success - VMKNETDDI_QUEUEOPS_OK Failure VMKNETDDI_QUEUEOPS_ERR

    Notes: A receive queue can support multiple filters.The kernel uses the VMKNETDDI_QUEUEOPS_OP_GET_FILTER_COUNT call to figure out how many receive filters are supported oneach queue. To apply a filter on a receive queue, VMkernelcalls VMKNETDDI_QUEUEOPS_OP_APPLY_RX_FILTER .

    vmknetddi_queueops_filter_t is used to specify a receivefilter. It is defined as:

    typedef struct vmknetddi_queueops_filter {vmknetddi_queueops_filter_class_t class;u8 active;

    union {unsigned char macadddr[ETH_ALEN];unsigned short vlan_id;struct {

    unsigned char macadddr[ETH_ALEN];

    unsigned short vlan_id;} vlanmac_t;

    } u;} vmknetddi_queueops_filter_t;

    class - Specifies the type of filter criteria.Class is an enumeration that can be either ofthe following:

  • 8/6/2019 Netqueues_1_3_2

    14/20

    NetQueue API 1.3

    14VMware Confidential

    VMKNETDDI_QUEUEOPS_FILTER_NONE - invalidfilter

    VMKNETDDI_QUEUEOPS_FILTER_MACADDR - mac addressfilter

    VMKNETDDI_QUEUEOPS_FILTER_VLANMACADDR- vlan tag + mac address filter

    If class is set to VMKNETDDI_QUEUEOPS_FILTER_MACADDR ,macadddr holds the MAC address the receive queue shouldallow traffic on.

    If class is set toVMKNETDDI_QUEUEOPS_FILTER_VLANMACADDR , macaddr andvlan_id fields of vlanmac_t hold the combined MACaddress and VLAN tag the receive queue should allowtraffic on.

    Example: mydriver_netqueue_ops(vmknetddi_queueops_op_t op, void *args){

    switch (op) {...case VMKNETDDI_QUEUEOPS_OP_APPLY_RX_FILTER:

    my_args = (vmknetddi_queueop_apply_rx_filter_args_t *)args;

    if (my_args->filter.class == VMKNETDDI_QUEUEOPS_FILTER_MACADDR) {

    my_qid = VMKNETDDI_QUEUEOPS_QUEUEID_VAL(my_args->queueid);

    my_fid = mydriver_enable_mac_filter(my_qid,my_args->filter.u.macaddr);

    my_args->filterid =

    VMKNETDDI_QUEUEOPS_MK_FILTERID(my_fid);}else {

    return VMKNETDDI_QUEUEOPS_ERR;}

    ....}

    }

  • 8/6/2019 Netqueues_1_3_2

    15/20

    NetQueue API 1.3

    15VMware Confidential

    VMKNETDDI_QUEUEOPS_OP_REMOVE_RX_FILTER

    Description: VMKNETDDI_QUEUEOPS_OP_REMOVE_FILTER removes a previouslyapplied receive filter on a queue

    Argument: typedef struct _ vmknetddi_queueop_remove_rx_filter_args_t {IN : struct net_device *netdev;IN : vmknetddi_queueops_queueid_t queueid;IN : vmknetddi_queueops_filterid_t filterid;

    } vmknetddi_queueop_remove_rx_filter_args_t;

    netdev: Pointer to struct net_device

    queueid : queue identifier of the allocated queue.The driver must useVMKNETDDI_QUEUEOPS_QUEUEID_VAL(queueid) to extractits internal queue id value

    filterid : filter identifier of the applied filter. The drivermust use VMKNETDDI_QUEUEOPS_FILTERID_VAL(filterid)

    to extract its internal filter id value

    Return values: Success - VMKNETDDI_QUEUEOPS_OK Failure VMKNETDDI_QUEUEOPS_ERR

    Example: mydriver_netqueue_ops(vmknetddi_queueops_op_t op, void *args){

    switch (op) {...case VMKNETDDI_QUEUEOPS_OP_REMOVE_RX_FILTER:

    my_args = (vmknetddi_queueop_remove_rx_filter_args_t *)

    args;my_qid =

    VMKNETDDI_QUEUEOPS_QUEUEID_VAL(my_args->queueid);my_fid =

    VMKNETDDI_QUEUEOPS_FILTERID_VAL(my_args->filterid);mydriver_disable_mac_filter(my_qid, my_fid);return VMKNETDDI_QUEUEOPS_OK;

    ....}

    }

  • 8/6/2019 Netqueues_1_3_2

    16/20

    NetQueue API 1.3

    16VMware Confidential

    VMKNETDDI_QUEUEOPS_OP_GET_STATS

    Description: VMKNETDDI_QUEUEOPS_OP_GET_STATS gets statistics associatedwith a queue

    Argument: typedef struct _vmknetddi_queueop_get_stats_args_t {IN : struct net_device *netdev;IN : vmknetddi_queueops_queueid_t queueid;OUT: struct net_device_stats *stats;

    } vmknetddi_queueop_get_stats_args_t;

    netdev: Pointer to struct net_device

    queueid : queue identifier of the allocated queue.The driver must useVMKNETDDI_QUEUEOPS_QUEUEID_VAL(queueid) to extractits internal queue id value

    stats : queue statistics

    Return values: Success - VMKNETDDI_QUEUEOPS_OK Failure VMKNETDDI_QUEUEOPS_ERR

    Example: mydriver_netqueue_ops(vmknetddi_queueops_op_t op, void *args){

    switch (op) {...case VMKNETDDI_QUEUEOPS_OP_GET_STATS:

    my_args = (vmknetddi_queueop_get_stats_args_t *) args;

    my_qid = VMKNETDDI_QUEUEOPS_QUEUEID_VAL(my_args->queueid);

    my_args->stats.rx_packets = driver_stats[my_qid].rx_packets;

    my_args->stats.rx_bytes = driver_stats[my_qid].rx_bytes;

    ...

    return VMKNETDDI_QUEUEOPS_OK;....

    }}

  • 8/6/2019 Netqueues_1_3_2

    17/20

    NetQueue API 1.3

    17VMware Confidential

    Frame Reception

    Description: On frame reception, the driver must inform the VMkernel the

    queue the frame was received on by callingvmknetddi_queueops_set_skb_queueid ():

    vmknetddi_queueops_set_skb_queueid (struct sk_buff *skb,vmknetddi_queueops_queueid_t cid);

    Arguments: skb: struct sk_buff describing the received packet

    cid: queue id of the queue the packet was received on.The driver must call VMKNETDDI_QUEUEOPS_MK_RX_QUEUEID() to convert its internal queue id value to one that canbe passed to the VMkernel

    Example: mydriver_intr_handler(...){

    ...for (my_qid=0; my_qid < MAX_CID; my_qid++) {

    if (mydriver_pkt_ready(my_qid)) {...skb = mydriver_get_next_pkt_from_queue(my_qid);

    vmknetddi_queueops_set_skb_queueid (skb,VMKNETDDI_QUEUEOPS_MK_RX_QUEUEID(my_qid);

    ...netif_rx(skb);...

    }}...

    }

  • 8/6/2019 Netqueues_1_3_2

    18/20

    NetQueue API 1.3

    18VMware Confidential

    Miscellaneous Netqueue API calls must only be compiled if __VMKNETDDI_QUEUEOPS__ is

    defined. Typically, all Netqueue specific code should be wrapped underthe following #define:

    #if defined(__VMKERNEL_MODULE__) && defined(__VMKNETDDI_QUEUEOPS__)...

    #endif

    The driver must take precaution to make sure shared data structures arelocked properly. VMkernel NetQueue calls are made from user context.Any data structures that are shared with interrupts and/or otherthreads must be properly locked.

    The driver must be prepared to handle duplicateVMKNETDDI_QUEUEOPS_OP_APPLY_RX_FILTER requests. If the filter is

    already applied on another (or the same) queue, the driver must returnan error.

    The driver must reload its queue and filter states if for any reason,the NICs internal state has to be reset. The current version of theAPI does not have a callback mechanism to notify VMkernel that theNICs internal state has been lost. Future versions of the API will addfunctionality to that effect.

  • 8/6/2019 Netqueues_1_3_2

    19/20

    NetQueue API 1.3

    19VMware Confidential

    Appendix----FAQ

    Q1: Are we permitted to deliver to the default queue packets for whichVMkernel has asked a filter to be installed?A1: No. This is a hard requirement. If a filter is in effect on another queue,delivering matching packets to the default queue can break things. If notanything else, it can lead to packets getting reordered. It might work in thepresent release, but things can/will break in future ESX releases.

    Q2: Is it permissible to deliver a packet to a non-default queue for which thefilter does not actually match?A2: Again, no. This is a hard requirement too.

    Q3: What are the most common filters? Is MACADDR most common, then maybeVLANMACADDR then VLAN?

    A3: MACADDR is the most common. VLANMACADDR is next. Were deprecating VLANfilters. The original goal of VLAN filters was to allow fanout along vlanboundaries, but MACADDR and VLANMACADDR are sufficient for the task of fanningout traffic. However, there is one use case that VLAN filters come in handy forand that is - promiscuous traffic reflection. For that to work, we'll have todefine a filter for promisc traffic reflection on untagged packets. We will takeour partner's input/feedback and consider it for the next API spec revision.

    Q4: How does promiscuous mode play into NetQueue? Does "promiscuous off"imply that packets not matching any filter and not destined for this NIC aredropped, or placed into the default queue?A4: We always put the nic in promisc mode. Promisc mode or not, there shouldbe no difference in the way the hardware filters packets.

    Q5: With VLANMACADDR filters, ethertype must be 0x8100 and the appropriateelements must match?A5: We do care for 802.1Q tagged packets, but we don't care about the embeddedethertype. VLANMACADDR can only match 802.1Q tagged packets.

    Q6: If I have filter of type MACADDR and a VLANMACADDR packet arrives with amatching MAC addr, does it match the MACADDR filter or the VLANMACADDR? It ispossible to have a packet match multiple filters? e.g., one filter might beVLANMAC=V1:M1 and another might be MACADDR=M1 and any packet with V1:M1 wouldmatch both?

    A6: Were deprecating VLAN only filters. The rule concerning MACADDR andVLANMACADDR filters is:

    A filter with only mac address, say MAC-X is equivalent to havinga filter with MAC-X + vlan id 0 (i.e., untagged vlan). For example:

    Queue 1: FILTER_VLANMACADDR{MAC-X, VLAN-A}Queue 2: FILTER_MACADDR{MAC-X}

    - Packets with MAC-X and VLAN-A should be directed to Queue 1.- Packets with MAC-X and no vlan should be directed to Queue 2.

  • 8/6/2019 Netqueues_1_3_2

    20/20

    NetQueue API 1.3

    20

    - All other packets, including those with MAC-X + VLAN(!A) go to thedefault queue.The NIC or the driver can always reject a filter it can not honor it.If a filter is successfully applied, ESX depends on it working correctly.

    Q7: If some packets may match multiple queues, should I just push them ontothe default queue?A7: The driver should fail the filter application if that can lead to such aconflicting situation.

    Q8: Do we do filtering in driver or hardware?A8: We don't want the driver to do any filtering. The filters should be actedupon in hardware.

    Q9: Is it possible for a VM to request more than one Netqueue? i.eDoes a many-to-one relationship exist from VM to Netqueue or a does amany-to-many relationship exists between them?A9: Yes. Many-to-Many. Also, queues can overlap between VMs.

    Q10: Will VMKernel use the netqueue ID set using set_skb_queueid()to direct the packet to the appropriate VM?A10: Netqueue ID can be used only if there is only one VM for a given queue, butthat is not guaranteed since there are cases that multiple VMs are using asingle queue. So setting queue id is not enough to demux packets in the case ofoverloaded queues. The use of netqueue is to fan out receive processing tomultiple CPUs and not to get rid of mac address lookups. In the future, thoughwe may support queue-VM bindings.