intermittent packet drops

Intermittent packet loss can waste an enormous amount of time. The network works, then it doesn’t. A Teams call turns choppy, an RDP session freezes for a few seconds, an application times out, or someone reports that “the network is slow”. By the time you start looking, everything appears normal again.

A common early mistake is assuming that every lost ping represents a lost packet affecting the application. Before replacing cables, changing QoS or blaming the ISP, you need to establish whether you’re actually seeing packet loss and, if you are, where it starts. This is the process I’d use:

  1. Work out whether the apparent loss is real or just ICMP being deprioritised.
  2. Prove the problem with sustained tests.
  3. Read the path correctly, rather than blaming the first hop that shows loss.
  4. Check the physical interface counters.
  5. Correlate the loss with other activity.
  6. Test for MTU problems.
  7. Capture traffic while the problem is actually happening.
  8. Understand how NIC offloading can distort a host-side capture.
  9. Escalate to the ISP with evidence.
  10. Validate the fix under the same conditions that caused the problem.

1. Is it actually packet loss?

This is the question I’d answer first. A common troubleshooting mistake is running ping 10.10.1.1, seeing a few timeouts, and immediately concluding that the network is dropping packets. That isn’t necessarily what’s happening.

Routers and switches commonly treat traffic addressed to their own management or control plane differently from traffic they’re forwarding. ICMP responses can be rate-limited or deprioritised, so a router may ignore some pings while continuing to forward application traffic normally. That’s why loss to an intermediate device isn’t automatically evidence of loss through that device.

Take this example output:

text

HOST: workstation
                                  Loss%   Snt   Last   Avg  Best  Wrst
  1. 10.10.1.1                     18.0%  1000   1.2   1.1   0.8   4.2
  2. 10.10.2.1                      0.0%  1000   2.8   2.9   2.4   6.1
  3. 172.16.20.1                    0.0%  1000   8.4   8.7   7.9  14.2
  4. 203.0.113.10                   0.0%  1000  14.1  14.3  13.6  21.5

In this example, the loss shown at hop 1 looks alarming, but every later hop and the final destination show none. Compare it with this second example:

text

HOST: workstation
                                  Loss%   Snt   Last   Avg  Best  Wrst
  1. 10.10.1.1                      0.0%  1000   1.2   1.1   0.8   4.2
  2. 10.10.2.1                      0.0%  1000   2.8   2.9   2.4   6.1
  3. 172.16.20.1                   12.0%  1000   8.4   8.7   7.9  14.2
  4. 203.0.113.10                  12.0%  1000  14.1  14.3  13.6  21.5

Here the loss begins at hop 3 and carries through to the destination, which is much stronger evidence that something is happening around that point in the path. The lesson is simple: loss at an intermediate hop that doesn’t continue to later hops isn’t, on its own, proof of forwarding loss.

2. Prove the problem with sustained tests

Once you know what you’re looking for, generate enough data to see the pattern. On Windows:

cmd

ping -n 5000 -l 1400 10.10.1.1

-n 5000 sends 5,000 requests and -l 1400 sends a 1,400-byte ICMP data payload. For a basic test, ping -n 5000 10.10.1.1 uses the default payload size.

The destination matters. Test something local first, then progressively further away:

text

Client
  |
  +-- Default gateway
  |
  +-- Local server
  |
  +-- WAN router
  |
  +-- Remote site
  |
  +-- Internet destination

If the client loses packets to its own gateway, that points in a very different direction from loss that only appears to an internet destination.

Use pathping on Windows

For a Windows path test:

cmd

pathping -q 50 -p 250 10.10.1.1

-q 50 sends 50 queries to each hop. Microsoft’s pathping reference gives the default as 100 queries, so specifying 100 wouldn’t change anything. -p 250 sets the wait between consecutive pings to 250 milliseconds, which is also the default.

Be patient with pathping. It first discovers the route and then spends time collecting statistics, and Microsoft notes that the interval between two pings to the same hop is the period multiplied by the number of hops. On a 10-hop path with the defaults, that’s 100 queries at 2.5-second intervals per hop, or roughly four minutes. If someone starts pathping, waits 30 seconds and kills it because they think it’s hung, they haven’t collected the useful part of the test.

When it finishes, look at the “This Node/Link” column rather than only the totals. Pathping estimates loss at each router separately from loss on the link leading to it, which helps you tell a router that’s ignoring pings apart from a link that’s genuinely dropping traffic.

MTR on Linux and macOS

MTR is well suited to longer-running tests:

bash

mtr -r -c 1000 10.10.1.1

-r produces a report and -c 1000 sends 1,000 probes. Don’t just look at the percentage. Look at where the loss begins and whether it continues to the destination.

3. Read MTR output properly

MTR becomes particularly valuable when the problem is intermittent, because it gives you a view of loss and latency across the whole path. Consider this example output:

text

HOST: PC01
                                    Loss%   Snt   Last   Avg  Best  Wrst
  1. 192.168.1.1                      0.0%  1000   1.0   1.1   0.7   4.5
  2. 10.20.0.1                       35.0%  1000   4.2   4.1   3.8   8.9
  3. 10.20.10.1                       0.0%  1000   6.2   6.3   5.7  11.2
  4. 203.0.113.1                      0.0%  1000  18.4  18.5  17.9  25.1

It’s tempting to look at hop 2 in this example and declare that the router is losing a third of its packets, but this output alone doesn’t justify that conclusion. If hop 2 is simply limiting its own ICMP responses while forwarded traffic carries on normally, the apparent loss disappears at the next hop, exactly as it does here. Compare that with a pattern where the loss persists:

text

Hop 1     0%
Hop 2    35%
Hop 3    35%
Hop 4    35%

Now you have much stronger evidence that the problem starts around hop 2 or on the link beyond it. MTR isn’t a magic tool that tells you which router is broken. It gives you evidence that has to be interpreted alongside the rest of the path.

4. Check the physical layer

Once you’ve established that the loss is real, look at the interfaces. On a Cisco switch, show interfaces GigabitEthernet1/0/10 gives the full detail for one port. For a quick check of the error and drop lines on that port, then a summary across the whole switch:

text

Switch# show interfaces GigabitEthernet1/0/10 | include error|drops|CRC
Switch# show interfaces counters errors

The summary might look like this:

text

Port        Align-Err  FCS-Err  Xmit-Err  Rcv-Err  UnderSize  OutDiscards
Gi1/0/10           0      127         0      127          0            0
Gi1/0/11           0        0         0        0          0          842
Gi1/0/12           0        0         0        0          0            0

These counters don’t all mean the same thing, and the difference sends you down very different paths.

CRC and FCS errors

Increasing CRC or FCS errors point towards the physical path. Investigate the patch leads, structured cabling, patch panels, SFPs and transceivers, NICs and interface hardware, and on older equipment, duplex mismatches. The important word is increasing. A port that has logged 127 CRC errors over its lifetime is different from one accumulating another 127 while the user is experiencing loss.

Input errors

Input errors are a broader category that covers several different conditions. Don’t look at a single input errors figure and conclude the interface is receiving damaged packets. Drill into the individual counters first.

Output drops

Output drops point somewhere else entirely:

text

GigabitEthernet1/0/10 is up, line protocol is up
  Input queue: 0/2000/0/0
  Total output drops: 18472

Output drops happen when traffic competes for an interface and the available output buffering runs out. A faster interface feeding a slower one, or several sources converging on one link, can create that condition. So the two counters lead to two different investigations:

text

CRC/FCS errors  ->  investigate the physical layer
Output drops    ->  investigate congestion, utilisation and traffic patterns

5. Correlate the loss with what else is happening

Intermittent problems often become much easier once you stop looking only at the network. Suppose the loss happens every night between 11pm and midnight. What happens at 11pm? Look at backup jobs, cloud synchronisation and replication, large file transfers, security scans, software deployments, VM backups and scheduled updates.

Record the exact times of the loss and compare them against your monitoring systems:

text

22:55   Normal traffic
23:00   Backup starts
23:04   WAN utilisation reaches 95%
23:06   Packet loss begins
23:21   Packet loss peaks
23:42   Backup finishes
23:43   Packet loss disappears

That’s far stronger evidence than saying “the network seems to get worse at night”. If the WAN interface also starts accumulating output drops at the same time, you have several independent pieces of evidence pointing at congestion.

6. Test for MTU problems

MTU problems produce some strange symptoms. Small pings work but large packets fail, some websites load while others don’t, VPN traffic behaves differently, or one application consistently times out. On Windows, test an IPv4 path with:

cmd

ping -f -l 1472 8.8.8.8

-f sets the IPv4 Don’t Fragment flag, and -l sets the ICMP data size. Windows then adds an 8-byte ICMP header and a 20-byte IPv4 header, so 1,472 + 8 + 20 = 1,500. That tests whether a normal 1,500-byte IPv4 packet can cross the path without fragmentation. On Linux:

bash

ping -M do -s 1472 8.8.8.8

If the larger packet fails, step the payload down until it succeeds:

cmd

ping -f -l 1464 8.8.8.8
ping -f -l 1452 8.8.8.8
ping -f -l 1400 8.8.8.8

If smaller packets succeed while larger ones fail, investigate the path MTU. This is especially relevant across IPsec VPNs, GRE tunnels, MPLS, SD-WAN, cloud connectivity and other encapsulated links. Path MTU Discovery also relies on ICMP messages telling the sender that fragmentation is needed. Filtering those messages creates the classic situation where small packets work but larger application traffic doesn’t.

7. Capture traffic when the problem is happening

When the basic tests don’t explain the problem, take a packet capture while it’s happening. Wireshark is particularly useful for TCP-based applications, and its TCP Analysis documentation explains each flag in full.

Filters that indicate loss

  • tcp.analysis.retransmission flags a segment Wireshark has already seen in that flow being sent again.
  • tcp.analysis.fast_retransmission flags a retransmission triggered by duplicate ACKs rather than a timeout. It’s the faster recovery path, and Wireshark labels it separately from an ordinary retransmission.
  • tcp.analysis.duplicate_ack shows the receiver re-acknowledging the last in-order byte, which signals a gap in the stream.
  • tcp.analysis.out_of_order flags a segment that arrived while earlier ones were still missing. This can indicate reordering rather than loss.

Filters that need more caution

This distinction catches people out, and getting it wrong sends you chasing a fault that doesn’t exist.

  • tcp.analysis.lost_segment is what Wireshark shows as “previous segment not captured”. It means a gap appeared at the capture point. The segment might have been lost upstream, might be arriving late, or might simply not have been captured, which is common at the start of a capture.
  • tcp.analysis.ack_lost_segment flags an ACK for a segment that never appeared in the capture. Again, it’s common at capture start, and often a sign that the capture device couldn’t keep up with the traffic rate rather than evidence of a network fault.
  • tcp.analysis.spurious_retransmission means the sender retransmitted a segment that had already been acknowledged, which usually points to an aggressive retransmission timeout rather than actual loss.

So a capture full of lost_segment entries isn’t automatically a network problem. Check whether the capture started mid-conversation, and whether your capture point could physically keep up, before you draw conclusions.

Capture at more than one point

A large number of retransmissions is useful evidence, but it doesn’t tell you where the packet was lost. If you can, capture at more than one point:

text

Client
  |
  | Capture 1
  |
Switch
  |
Firewall
  |
WAN
  |
  | Capture 2
  |
Server

If a segment appears in the first capture but not in the second, you’ve narrowed down where to investigate.

8. Why your capture shows a 12,000-byte TCP segment on a 1500-MTU link

This catches people out when they first analyse traffic on a server itself. You may see a frame length of 12,000 bytes even though the interface has a 1,500-byte MTU. That doesn’t necessarily mean someone is sending 12,000-byte Ethernet frames across the network.

Features such as TCP Segmentation Offload (TSO), Large Send Offload (LSO), Generic Segmentation Offload (GSO) and Generic Receive Offload (GRO) let the operating system or NIC work with larger buffers before traffic is split into wire-sized packets. A host-side capture can therefore show traffic that doesn’t correspond directly to individual packets on the wire. Checksum offloading has a similar effect, often making outgoing packets appear to have bad checksums in a local capture.

If you suspect an offload-related capture issue, temporarily disabling the relevant feature is a useful diagnostic test, but it isn’t automatically the permanent fix. Offloading exists to reduce CPU overhead and improve performance. If disabling it makes the capture look normal, investigate the NIC driver, operating system version and hardware compatibility, rather than leaving every offload feature switched off in production. If you need to know exactly what went onto the wire, a capture from another system, a SPAN port or a network tap is more representative than one taken on the endpoint itself.

9. Escalate to the ISP with evidence

If your testing points outside your network, don’t send the ISP a ticket saying “we’re getting packet loss”. Give them something they can investigate: source and destination IPs, exact timestamps with the time zone, the loss percentage and latency, whether the problem is continuous or intermittent, tests from more than one source if you have them, MTR or pathping results, and your WAN interface utilisation, errors and drops. For example:

text

22/09/2026 14:05-14:17 AEST

Source:       10.20.10.50
Destination:  203.0.113.50

Observed:     8-12% loss
Normal:       0% loss

MTR:          Loss begins after the provider handoff and continues to destination.

Local WAN interface:
              0 CRC errors
              0 input errors
              0 output drops
              No local saturation observed

That gives the provider something concrete to work with. For intermittent problems, the timestamps matter most, because the provider needs to correlate your evidence against its own circuit and interface statistics.

10. Validate the fix

This is where network troubleshooting often stops too early. Someone changes something, the user says “it seems better”, and the incident gets closed. That isn’t really validation. If the problem originally happened under a particular condition, reproduce or monitor that condition after the change:

text

Before change:
23:00-23:30   8% packet loss
              WAN utilisation 95%
              Output drops increasing

Change:       Backup traffic moved to another window

After change:
23:00-23:30   0% packet loss
              WAN utilisation 62%
              0 output drops

That’s a much stronger result. For a physical fault, record the interface counters before and after testing. If you need a clean baseline, Cisco provides clear counters GigabitEthernet1/0/10, but record the existing values first. Clearing counters destroys the history you may need to understand the problem, and on production infrastructure other people may rely on those counters for their own monitoring and troubleshooting.

After setting your baseline, monitor the interface while the problem conditions occur. The aim isn’t just to make the symptom disappear. It’s to show that the evidence behind the original diagnosis has disappeared too.

What intermittent packet loss really comes down to

The hardest part of intermittent packet loss isn’t knowing that ping, MTR, Wireshark and switch counters exist. It’s knowing what the results actually mean. A percentage beside a hop in MTR isn’t automatically packet loss. A CRC counter isn’t the same problem as an output-drop counter. A TCP retransmission proves something went wrong with the flow, but not where. A lost_segment flag may mean your capture missed the packet rather than the network dropping it. A large TCP segment in a host capture doesn’t mean a large packet went onto the wire. And disabling a NIC offload feature might prove a driver or capture interaction is involved, but that doesn’t make disabling it the permanent fix.

Work through it as evidence rather than guesswork, and you end up with something far more useful than a list of possible causes: an explanation of exactly what happened, and why the eventual fix worked.

Intermittent packet loss checklist

  1. Ping the local gateway with ping -n 1000 10.10.1.1 while the problem is happening. Loss here points at the client, its cabling or the access switch rather than anything further away.
  2. Ping a known local resource, such as ping -n 1000 10.10.1.20. If the gateway is clean but this isn’t, look at that server and its connection.
  3. Ping across the WAN, such as ping -n 1000 10.20.1.1. Loss that only appears here moves the investigation to the WAN link and beyond.
  4. Run pathping -q 50 -p 250 10.20.1.1 on Windows or mtr -r -c 1000 10.20.1.1 on Linux and macOS, let it finish, and find the first hop where loss starts and carries through to the destination.
  5. On the switch ports along that path, run show interfaces ... | include error|drops|CRC and show interfaces counters errors, and note whether CRC/FCS errors or output drops are increasing.
  6. Test the path MTU with ping -f -l 1472 on Windows or ping -M do -s 1472 on Linux, stepping the size down if it fails.
  7. Correlate the loss timestamps with WAN and firewall utilisation, interface drops, backups, wireless events, device CPU and application monitoring.
  8. Capture the affected traffic, ideally at two points, and look for retransmissions, fast retransmissions, duplicate ACKs and out-of-order segments. Treat lost_segment, ack_lost_segment and spurious retransmission flags with caution.
  9. If the capture is from the endpoint itself, account for TSO, LSO, GSO, GRO and checksum offloading before interpreting odd packet sizes or checksums.
  10. After making the change, re-test under the same conditions that caused the problem, and only close the incident once the original evidence has gone.

References

Leave a Reply

Your email address will not be published. Required fields are marked *