Hyper-V 10G connections and File Transfer speed


for the past two weeks, I’ve been bashing my skull around configuring a working SET with a HyperV cluster, to no avail.

The systems are two Lenovo SR530 with 2x10G Intel x722 NICs, hosts are running 2016, guests are Windows 10 1809. 

The NICs FW and all other firmware is fully updated. The drivers are installed through Lenovo XClarity Essentials UpdateExpress and there are no more driver updates available. 

There is no antivirus or firewall on neither the hosts, nor the VMs.

After some pain, I decided to go to the sheer basics and configure:

  • Initial test with no vSwitch, just two NICs and a direct speed test and then everything below
  • iPerf host-to-host with no vSwitch configured, just a direct 10G link 
  • iPerf tests at 9.5 Gbps (great, so the NICs are working at 10G-ish)
  • Copying files between hosts works at ~700 MB/s 

Subsequent tests with a vSwitch (configuration):

  • A single 10G network adapter on each host (all other NICs disabled)
  • A single vSwitch on each host created through HyperV Manager with SR-IOV enabled, AllowManagementOS is Enabled.
  • All VM interfaces are with default settings, meaning VMQ and SR-IOV are enabled
  • A single VM on each host

The tests:

  • iPerf host-to-host with a vSwitch enabled, 10 streams (-P10 switch)
  • Copy a 4,5 GB ISO file host-to-host
  • iPerf guest-to-guest with a vSwitch, 10 streams (-P10 switch)
  • Copy a 4,5 GB ISO file guest-to-guest

The results are:

  • 4.5 Gbps (OK, I guess it’s VMQ doing something)
  • 260 MB/s stable transfer (not ideal, but OK)
  • 4.5 Gbps (OK, seems stable)
  • 10-60 MB/s unstable transfer (What-the-heck?!)

When I transfer all of this workload on the 1G NICs, eveything is working full-speed.

Additionally, when I transfer the same file from a VM on a 3rd host, which only has 1G network, to my VM which has 10G, everything is stable at 1G (line speed of the sender station).

That brings me to the conclusion that there is something very wrong with my 10G cards when they communicate with each other, but that’s as far as I can go, because I have no other machines to test 10G externally.

Bottom line - as soon as a vSwitch comes into play, the guest-to-guest file transfer speed drops to 0.3 Gbps.

So, please, any suggestions as to what I may be doing wrong or am missing would be highly valued.

Read full post . . . .

Go Que Newsroom Categories

Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to allocate 8192 bytes) in /home/content/36/8658336/html/goquecom/wp-includes/taxonomy.php on line 3363