Broadcom QLogic VMQ Hyper-V Network Bug

UPDATE August 2017, following this issue as it’s in my top 10 most viewed posts. User @StephanF over on reports that the issue persists with newer 10G Broadcom/QLogic NIC’s. “Updates of drivers, NIC firmware etc. did not help … disabled VMQ on all interfaces using PowerShell … Since then the issue is gone.”

UPDATE May 2016, seeing similar problem in the wild this year with NEW Dell PowerEdge R730 server Broadcom/QLogic 1Gbe NIC, Server 2012 R2 OS. NEITHER Hyper-V NOR Teaming were used. Latest Dell OMSA (OpenManage Server Administrator GUI) reported healthy system (no critical firmware updates needed). Get-NetAdapterVmq reported $False for all NICS. ADVANCED NIC SETTINGS showed BOTH VMQ AND FLOW CONTROL were enabled within the NIC driver – DISABLING BOTH resolved the issue for us immediately on this system. Problem showed up ONLY under network load (robocopy command, seeding new file server from old file share across network).

Broadcom / QLogic NIC’s on Hyper-V servers can cause significant network performance issues for both the host and guest virtual machines. There is a potential to resolve the issue with driver and firmware updates if a fix has been released by your hardware manufacturer. For many users the solution seems to be a workaround to disable VMQ for all Broadcom network interfaces and teams in the system.

Microsoft, Dell, and numerous other hardware/software vendors have confirmed the issue and suggested workarounds. If you’re running Windows Server 2012 R2, I recommend using the new Powershell Get/Set-NetAdapterVMQ commands.

  • Get-NetAdapterVmq # look for problem broadcom adapters or teams with VMQ true (enabled)
  • Set-NetAdapterVmq -Name NIC1,NIC2,NIC3 -Enabled $False
  • Get-NetAdapterVmq # confirm that problem adapters have changed to VMQ false (disabled)

Another suggestion that has been repeated often for this type of issue – consider switching your Hyper-V network adapters to Intel or other hardware manufacturer with better Hyper-V compatibility history.

Related Articles:


About notesbytom

Keeping technology notes on to free up my mind to solve new problems rather than figuring out the same ones repeatedly :-).
This entry was posted in Networking, System Administration and tagged , , . Bookmark the permalink.

9 Responses to Broadcom QLogic VMQ Hyper-V Network Bug

  1. Brett Smith says:

    This is invalid. VMQ is a feature that works with Broadcom drivers, but some of the prerequisites are that you only use Windows Server 2012 NIC Teaming Load balancing options (Address Hash or Hyper-V port). VMQ doesnt work when Dynamic is selected in Windows Server 2012 R2.

    HP and Broadcom released driver version 17 for Broadcom NetXtreme Adapters which allows VMQ to work correctly with their NICs.

    • notesbytom says:

      From your statement it seems that both (a) broadcom drivers in the wild without a patched version like you mention have a vmq issue, and (b) the default “dynamic” team setting in win srv 2012r2 has a vmq issue.
      Do you have a link or reference regarding the server 2012r2 dynamic team vmq issue? Is the v17 update supposed to fix the dynamic team load balancing vmq issue, or just other vmq issues?
      We experienced this exact issue (referenced in above blog post) on a recent model Dell Hyper-V server with Win 2012 R2 and “dynamic” load balancing default on broadcom team. The powershell commands above resolved the issue (we didn’t have a chance to try alternate NIC driver versions).

  2. Pingback: High latency to and from Hyper-V guests with VMQ – Cook UoP

  3. Pingback: High latency to and from Hyper-V guests with VMQ – Adam Cook

  4. Hans V says:

    I have exact the same problem describe above. “Problem showed up ONLY under network load (robocopy command, seeding new file server from old file share across network).”
    But Disable BOTH VMQ AND FLOW CONTROL dit not solve the problem for me.

    Windows 2016 server, Broadcom 57416 Dual Port 10Gb.

    • notesbytom says:

      Hello @Hans-V, you might want to go to the advanced properties for the Network Interfaces (NIC’s) and disable settings related to Chimney, Scaling/RSS, Offload, etc. Sometimes trouble with these hardware optimizations can cause similar issues.

      • Hans V says:

        I have disable everyting with VMQ, Scaling, RSS and Offload. But nothing helps. I have now asket to DELL to exchange the Boadcom with Intel NIC’s.

  5. Hans V says:

    We have discovered the following. Network works well after al cold boot. After a warm reboot the problems are back.
    Network is hanging under network load (robocopy command, seeding new file server from old file share across network).

  6. Hans V says:

    We replaced the Broadcom NIC with “Intel X550 Dual Port 10G Base-T Adapter” NIC. Everything works well.
    Broadcom in combination with Hyper-V is a bad combination. We also had problems with Broadcom three years ago. I had hoped they had solved the problem. Not so!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s