NOTE: You can now take course by the author with video and example traces on Wireshark. Check this post for more details.

Author’s Note: This is the fourth part in a six-part series about finding and solving many networking anomalies using the Wireshark network protocol analyzer. If you are new to the series, you can find part 1 here, and the whole series here.
Application layer inspection refers to a technique employed by many firewalls and some routers whereby the application layer traffic is inspected for common exploits. Some of the products that perform this inspection only look for known exploits, while others will reject traffic that is simply not strictly RFC compliant. The bottom line is that when one of these devices locates a non-compliant packet, it will reject the packet, and in many cases, will forcibly tear down the session. As with most networking problems, the key to detecting this issue is in obtaining simultaneous traces while reproducing the issue.
Perhaps the most common culprit of these problems are Cisco PIX firewalls, which have several features that can disrupt non-RFC compliant traffic. In some cases, a PIX will forcibly tear down non-compliant sessions by sending a reset to end the session. This reset will appear to come from the remote host in the session, but in actuality is generated from both sides by the PIX. One of the many issues caused by this feature is documented in KB 911786. When presented with these problems, filter down to the two hosts involved in the session using the “ip.addr ==” filter and look for missing traffic from both sides of the connection. Also, check for session resets (the RST flag in your TCP header) that appear to be coming from one side of the connection, but do not showing up as being sent from the system on that side.