Fix Large packet loss at the guest level on the VMXNET3 vNIC in ESXi Using Powershell

Problem:-

Most of SQL Server, today hosted on VMware, especially if that Server host very critical databases with high OLTP. When using the VMXNET3 driver on ESXi 4. x, 5. x, 6. x There is a known issue on the VMware regarding the network communications. VMware published an article on this issue. to change the Small Rx Buffer size and Rx Ring #1 Size on the NICs. you see significant packet loss during periods of very high traffic. 

also, the obvious problem if we need to apply that configuration at a very large number of servers, as usual, the answer is Powershell 😊

Solution:-

First:-

We need to get Current Setting by running below script for all ‘UP’ NIC Cards that meaning Running NICs if Server had more than NIC.

#Report About Current Setting Locally for both setting Small Rx Buffers & Rx Ring #1 Size

Get-NetAdapterAdvancedProperty (Get-NetAdapter | where status -eq 'Up' | select -ExpandProperty name) -DisplayName "Small Rx Buffers"-Verbose

Get-NetAdapterAdvancedProperty (Get-NetAdapter | where status -eq 'Up' | select -ExpandProperty name) -DisplayName "Rx Ring #1 Size" -Verbose

#Get Configuration for more server listed in TXT file (IP’s) using PS Remoting

Invoke-Command -ComputerName (Cat "C:\Users\Mk.elsawy\Desktop\Servers.txt") -ScriptBlock { Get-NetAdapterAdvancedProperty (Get-NetAdapter | where status -eq 'Up' | select -ExpandProperty name) -DisplayName "Small Rx Buffers"-Verbose} -Credential (Get-Credential)

Invoke-Command -ComputerName (Cat "C:\Users\Mk.elsawy\Desktop\Servers.txt") -ScriptBlock {Get-NetAdapterAdvancedProperty (Get-NetAdapter | where status -eq 'Up' | select -ExpandProperty name) -DisplayName "Rx Ring #1 Size" -Verbose} -Credential (Get-Credential)

Report Current Small Rx Buffers & Rx Ring #1

Second:- Applying VMware fix by running below scripts

*Note:-

These changes will happen on the fly, so no reboot is required. However, any application sensitive to TCP session disruption can likely fail and have to be restarted. This applies to RDP, so it is better to do this work in a console window.

#Change Setting Locally Based on VMware Best Practice for all Running NICs

Set-NetAdapterAdvancedProperty (Get-NetAdapter | where status -eq 'Up' | select -ExpandProperty name) -DisplayName "Small Rx Buffers" -DisplayValue "8192" –NoRestart -Verbose

Set-NetAdapterAdvancedProperty (Get-NetAdapter | where status -eq 'Up' | select -ExpandProperty name) -DisplayName "Rx Ring #1 Size" -DisplayValue "4096" –NoRestart  -Verbose

Set Small Rx Buffers

Set Rx Ring #1 Size

#Change Configuration for more server listed in TXT file (IP’s) using PS Remoting

Invoke-Command -ComputerName (Cat "C:\Users\Mk.elsawy\Desktop\Servers.txt") -ScriptBlock {Set-NetAdapterAdvancedProperty (Get-NetAdapter | where status -eq 'Up' | select -ExpandProperty name) -DisplayName "Small Rx Buffers" -DisplayValue "8192" –NoRestart -Verbose} -Credential (Get-Credential)

Invoke-Command -ComputerName (Cat "C:\Users\Mk.elsawy\Desktop\Servers.txt") -ScriptBlock {Set-NetAdapterAdvancedProperty (Get-NetAdapter | where status -eq 'Up' | select -ExpandProperty name) -DisplayName "Rx Ring #1 Size" -DisplayValue "4096" –NoRestart  -Verbose} -Credential (Get-Credential)

The case you need to Rollback:-

Set-NetAdapterAdvancedProperty (Get-NetAdapter | where status -eq 'Up' | select -ExpandProperty name) -DisplayName "Small Rx Buffers" -DisplayValue "512" –NoRestart -Verbose

Set-NetAdapterAdvancedProperty (Get-NetAdapter | where status -eq 'Up' | select -ExpandProperty name) -DisplayName "Rx Ring #1 Size" -DisplayValue "1024" –NoRestart  -Verbose

Using GUI: -

1. Click Start > Control Panel > Device Manager. 

2. Right-click vmxnet3 and click Properties. 

3. Click the Advanced tab. 

4. Click Small Rx Buffers and increase the value. The default value is 512 and the maximum is 8192. 

5. Click Rx Ring #1 Size and increase the value. The default value is 1024 and the maximum is 4096. 

Happy Troubleshooting 👊👍

Ref:-

https://kb.vmware.com/s/article/2039495