Issue:
One or more applications that are accessed over a VPN Tunnel do not work consistently. Typically the applications affected are web servers and web applications that run in a web browser. Other applications can be affected but are not as easily identified.
A typical web application experiencing an MTU Path Discovery issue may have the following symptoms:
1) A simple Ping to the web server works (ping ipaddress)
2) The initial page typically does not load and appears to hang for around 90 seconds before timing out.
3) If the initial page does load functionality is sporadic and periodic hangs of 90 seconds prior to error messages are typical.
4) Network captures from the client show HTTP requests to the server with few responses. Large gaps of time are spent waiting on replies from the server.
Cause:
By default every VPN Tunnel solution reduces the MTU of the tunnel by at least 100 bytes. The result is an MTU for the VPN Tunnel of 1400 bytes or less. The default MTU of Ethernet is 1500 bytes.
Windows 2000/2003/XP all implement Path MTU Discovery (RFC 1191). This is used to determine the maximum size packet that can be sent. Path MTU Discovery needs to work in both directions (client-to-server and server-to-client).
Path MTU Discovery relies on three basic underlying assumptions of a normal working TCP/IP protocol stack. The first is the ability to set the "Don't Fragment" (DF) flag in the IP header of any IP packet. The second is the ability of a device (router or end computer) to use the ICMP protocol to respond to the DF flag and notify the sender "Fragmentation needed but DF set" error. The third is for the sender of the IP original IP packet to receive the ICMP message.
Most issues relating to MTU Path Discovery are because ICMP is not properly implemented throughout the network (or at least end-to-end for the client-to-server communications affected.). This can include ICMP being completely or partially disabled on the client, server or any router in between.
Troubleshooting MTU Path Discovery issues may be further complicated because a VPN Tunnel can exist in the form of a Client-to-Lan or Lan-to-Lan configuration. To simplify troubleshooting explanations a Client-to-Lan scenario is assumed. If you have a Lan-to-Lan VPN tunnel in between your client and server, the basic principals for troubleshooting still work, but you may need to adjust your troubleshooting steps.
Diagnosing the problem:
Check for bad configuration
1) Because of the perceived security threat associated with ICMP some administrators disable ICMP on web servers. Check to see if this has been done in your environment. If so enable ICMP on the web server, and retest the application.
2) If you access the web server through a firewall, check to see that ICMP is enabled. This includes both allowing ICMP to pass through the firewall and allowing the firewall to accept and respond to ICMP packets sent directly to it. The firewall acts as a router and will need to respond to ICMP packets for Path MTU Discovery to work correctly.
Determine Maximum Packet Size (not the MTU):
1) The first step (on client and/or server) is to determine the Maximum Packet Size on the respective computer.
2) To do this simply use the following command until to you receive “Packet needs to be fragmented but DF set.”
a. Ping defaultgateway –f –l size
i. Defaultgateway: is the ip address of the gateway for the VPN tunnel on the client or the router on the server.
ii. size: is the size of the packet to send
1. Start with 1372 and increase or decrease until the message is received.
2. The highest value you can send without receiving the message is the Maximum Packet Size for that interface (maxsize).
iii. Make note of this value (maxsize), for use later.
iv. Example:
Maximum Packet Size: 1472
MTU: 1500 (1472 + 28)
b. By pinging the default gateway, you ensure you are finding the actual Maximum Packet Size that is limited by the MTU of the interface. By default all ping packets will have 28 bytes of header information. Simply add 28 bytes for actual MTU of the interface.
From the Client / Server (written for the client side, adjust accordingly for the server side.)
1) Ping servername
a. A response expected, if no response troubleshoot that issue first.
2) Trace route to servername
a. tracert servername
i. Record all router ip addresses
ii. If no response on the trace route, then troubleshoot that issue first.
3) Ping each router ip address with Maximum Packet Size
a. Ping ipaddress –f –l maxsize
b. If you receive “Reply from w.x.y.z: …”
i. Continue with the next router ip address. There is no issue with this router.
c. If you receive “Packet needs to be fragmented but DF set.”
i. Then there is a decrease in the MTU between the previous router and the current one being pinged.
ii. Adjust the maxsize value down until you determine the new maximum packet size allowed (newmaxsize). Continue with the next router ip address with this new maximum packet size.
iii. Make note of the original values (maxsize) and the new value (newmaxsize).
d. If you receive “Request timed out.”
i. Then the packet was successfully sent to the destination, but the reply has been blocked or dropped.
ii. The reply could have been blocked or dropped at the destination or at any router/firewall in the return path. Troubleshoot as needed to determine where ICMP messages may be failing.
iii. Specifically look for rules that prevent ICMP Type 3 Code 4 packets.
4) Ping the server ip address with the Maximum Packet Size.
a. Ping server –f –l maxsize (or newmaxsize)
b. If you receive “Reply from w.x.y.z: …”
i. If you are using newmaxsize or maxsie
1. The issue is most likely directly on the web server or the router just before it. Focus your troubleshooting efforts in these locations. Most likely ICMP is partially disabled. Specifically ICMP Type 3 Code 4 packets are being blocked.
c. If you receive “Packet needs to be fragmented but DF set.”
i. The MTU on the client and server do not match. Most likely because of the VPN Tunnel running on the client. If you get this message Path MTU Discovery should be working correctly, unless there is something blocking ICMP Type 3 Code 4 packets.
d. If you receive “Request timed out.”
i. Then the packet was successfully sent to the destination, but the reply has been blocked or dropped.
ii. The reply could have been blocked or dropped at the destination or at any router/firewall in the return path. Troubleshoot as needed to determine where ICMP messages may be failing.
iii. Specifically look for rules that prevent ICMP Type 3 Code 4 packets.
Client / Server Side Workaround:
This method is not recommending because it can greatly reduce the performance of the application. Adjust the client side first, if that does not work, then adjust the server side was well. This procedure is written for the client, adjust as needed for the server.
Use the newmaxsize value determined above to set the MTU for the VPN Tunnel interface.
1) On the Client
2) Start the VPN Tunnel and make a connection
3) Run IPConfig to determine the IP Address of the VPN connection
4) Run Regedit
5) Goto HKLM\System\CurrentControlSet\Services\Tcpip\Parameters\Interfaces
a. Locate the GUID key for the DhcpIPAddress setting that matches the one from IPConfig
i. Example:
b. Look for the MTU Setting in the GUID key
c. If MTU does not exist create as follows, if it does exist edit as follows
i. Key name: MTU
ii. Key Type: DWORD
iii. Key Value: newmaxsize + 28
1. Remember to add 28 bytes to the maxsize value to place the actual MTU size correctly.
6) Reboot the computer for the setting to take affect.
More Information:
Some administrators mistakenly think that they should disable the ICMP protocol on certain devices because of a desire to prevent Ping from working for added security. This is a very bad practice. ICMP is more that just ping, there are many features of ICMP that other protocols, applications, or processes rely on. You should never disable the ICMP protocol without fully understanding what will be affected.
Adjusting IP MTU, TCP MSS, and PMTUD on Windows and Sun Systems
http://www.cisco.com/en/US/tech/tk870/tk877/tk880/technologies_tech_note09186a008011a218.shtml
Read the “PMTUD Failure” section
Path MTU Discovery (RFC 1191)
http://www.ietf.org/rfc/rfc1191.txt