Java Lies! IOException

Recently a coworker and I ran into a strange IOException that was only occurring on our Linux servers, but Windows, OS X, and Solaris never had the problem. Our software monitors networking equipment via SNMP, which is a UDP-based protocol. We communicate via UDP at high rates of hundreds of devices per second. When testing a new version of our product we came across “IOException: No buffer space available”.

Turns out that this is a misleading exception from Java. The root cause is Linux has a hard limit of 1024 ARP entries in the local system. Since our server’s IP address was in the same subnet as all the target IP addresses (10.140.0.0/16), the local box issues ARP requests to translate the IP address into the hardware MAC address in order to transmit the UDP packet. After 1024 of these lookups are done the magical IOException occurs.

Turns out I’m not the only person to encounter this. While I was looking into the issue Google did not turn up much at all, except for this one StackOverflow post where I’ve posted my answer.