Uncovering virtual networking Part-9: IOChain Framework

The IOChain is a group of functions that provides connectivity between virtual ports and the virtual switch. It also provides the capability to insert functions into the network data-path regardless of the switch type used i.e. Standard Switch (vSS) or Distributed Switch (vDS).

A virtual port has two IOChains based on the direction i.e. to and from the vSwitch. Meaning each port is associated with an input and an output IOChain. Optional elements in an IOChain framework includes VLAN support, NIC teaming, and traffic shaping.

Standard Switch:

The following diagram shows all components of IO chain framework on a standard vSwitch:

Image: VMware

Distributed Switch

A vSphere distributed switch spans switch configurations over multiple ESXi hosts that can be grouped over multiple clusters. The network information is distributed to all member ESXi hosts which ensures network configuration consistency in vSphere environment.

Distributed switch provides quality control features like Network I/O Control (NIOC), ingress and egress traffic shaping and traffic flow monitoring options. A vDS also offers additional teaming options such as Load Balanced Teaming and LACP.

Image: VMware

As shown in the diagram above, we see the additional DVfilter components in vDS IOChain framework. The DVfilter in vDS is an API framework required for NSX. When NSX is installed, it introduces additional kernel modules in vSphere ESXi. The summarize-dvfilter command can be used to show the loaded DVfilter agents and filters per port.

As in above images, The ESXi network IOChain framework includes these three main levels:

•Port group

•Virtual switch (standard or distributed)

•Uplink

Port group level

This is where an optional component VLAN is interpreted by the VLAN filter, allowing for VLAN dot1q tags for your port group. The security settings like Promiscuous mode, MAC address changes, and Forged transmits are also set at the port group level. The user can also optionally configure traffic shaping. A standard switch supports only egress traffic shaping whereas a vDS supports bi-directional traffic shaping.

vSwitch (VSS or VDS) level

Incoming packets at the vSwitch level are forwarded to their destination using the forwarding engine. The forwarding engine contains port information paired with MAC address information.

The teaming engine is responsible for balancing network packets over the uplink interfaces based on the chosen teaming policy. The traffic shaper module is added to the IOChain if enabled in the port group level.

Uplink level

At this level, the traffic sent from the vSwitch to an external host finds its way to the driver module. This is where hardware offloading takes place. The Supported hardware offloading features depends on the physical NIC in combination with a specific driver module.

Typically supported hardware offloading functions that in NICs are TCP Segment Offload (TSO), Large Receive Offload (LRO) or Checksum Offload (CSO). Network overlay protocol offloading like with VXLAN and Geneve, as used in NSX-v and NSX-T respectively, are widely supported on modern NICs.

Next to hardware offloading, the buffer mechanisms come into play in the Uplink level. I.e., when processing a burst of network packets, ring buffers come into play. Finally, the bits transmit onto the DMA controller to be handled by the CPU and physical NIC onwards to the Ethernet fabric.

Leave a Reply