How’s your Dell for SR-IOV?

This post covers how to enable your Dell server hardware for Single Root I/O Virtualization (SR-IOV) capability, and how to configure SR-IOV for a VMware Virtual Machine.

The main reason for this post is that I was looking into SR-IOV after seeing how it was showing as Disabled when I was Adding a vSphere infrastructure to VMware Telco Cloud Automation.

SR-IOV is a common feature required in the Telco space, specifically for use with Virtualised Network Functions (VNFs). An SR-IOV capable NIC is required in order to provide VNFs with direct access to the physical network card, bypassing the network functionality in the hypervisor.

As per Dell KB000144029: “The Single Root I/O Virtualization (SR-IOV) specification details how a single PCIe device can be shared between various guest operating systems. Devices capable of SR-IOV functionality support multiple virtual functions on top of the physical function. The virtual function is enabled in the hardware as a light-weight PCIe function. The virtual function of an SR-IOV capable card can be directly assigned to the guest operating system. A virtual function driver running in the guest operating system manages this device.

Configuring SR-IOV in vSphere means configuring a passthrough device (VMDirectPath I/O) for a Virtual Machine, which in turn means a trade-off in other vSphere features for that VM. More here.

This post contains the following sections:

  • Check Hardware Compatibility
  • Configure server device hardware for SR-IOV
  • Configure vSphere ESXi hosts for SR-IOV
  • Configure virtual machine for SR-IOV

Check hardware compatibility

Check your Dell server to confirm that it is SR-IOV capable, by looking in the iDRAC, under Hardware > Network Devices. The system in use here, is a 13G Dell PowerEdge R730, using an Intel 10G 2P X520 Adapter for the SR-IOV functionality.

Also confirm VMware NIC support for SR-IOV @ VMware Compatibility Matrix

I used’ x520‘ as a Keyword to filter the search results a bit quicker!


Configure server device hardware for SR-IOV

Reboot the host, and on boot hit F2 to enter System Setup.

As shown in the slideshow below:

  • Select Device Settings and from there select the physical NIC to be configured for SR-IOV, which, in this example, is in Slot 5 – Intel 10G 2P X520 Adapter.
  • Select Device Level Configuration
  • Set Virtualization Mode to SR-IOV
    • This will apply the setting to both ports on this 2-port NIC
  • The change need to be saved, and will be implemented on next reboot of host

Configure vSphere ESXi hosts for SR-IOV

The next layer to configure for SR-IOV is the VMware layer, so via the vSphere client, we select and configure each ESXi host as follows:

  • Navigate to Configure > Networking > Physical Adapters and select the appropriate vmnic, which in this example if vmnic4 (one of a two-port 10Gb NIC). As per the adapter details summary, and the SDR-IOV Status column, SR-IOV is currently Disabled.
  • Click the Edit button for the vmnic
  • Set the Status of SR-IOV to Enabled
    • For this example we are just leaving the Number of virtual functions at the default value of ‘1’
  • We can repeat this for the second vmnic (if also required), so that both vmnics will be SR-IOV enabled.

Configure virtual machine for SR-IOV

The remaining step is to provide a virtual machine with an SR-IOV capable pass-through device. For a RHEL 6+ or Windows VM, the following steps can be followed:

  • Add a new Network Adapter
  • Configure the relevant PortGroup
  • Change Adapter Type to SR-IOV Passthrough
  • Confirm the correct vmnic for the Physical Function
  • Reserve all guest memory

Note also the warnings around features that are not supported or available when SR-IOV is in use (for example, vMotion, HA, FT, DRS, DPM, Snapshots, and more). Please refer to VMware’s SR-IOV Support documentation for more details.

In terms of functional performance of SR-IOV, Faisal Khan over on TelcoCloudBridge has a great breakdown of how SR-IOV works in comparison to DPDK (Data Plane Development Kit).

And that’s it for this post, that’s how to enable SR-IOV on a Dell PowerEdge server for VMware vSphere, hope that helps!

Resources

Advertisement

One thought on “How’s your Dell for SR-IOV?

  1. Pingback: Adding a vSphere infrastructure to VMware Telco Cloud Automation | Scamallach

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.