arpresolve: can't allocate llinfo

[Cory Albrecht]

Hello all,

This is becoming rather frustrating, and I'm hoping somebody can help me.

Every so often my OpenBSD stops routing packets and dmesg fills up with
"arpresolve: can't allocate llinfo". Before I go any further, let me
illustrate my network setup.

'Net---ADSL---WRT54G . . . (ral0)OpenBSD(em1)---Linux
. (em0)
. |
LapTop |
WinXP


Dashes are wired, dots are wireless.

Seemingly randomly, the OpenBSD just looses the ability to facilitate
new IPv4 connections via ral0. While IPv4 connectons from my laptop fail
(like share mounted form teh Linux Samba server), IPv6 ssh sessions
still work. If I try to ping the IP associated with ral0 (192.168.1.2) I
get no response.

If I go the the console of the OpenBSD machine, I can ping the WRT54G
(192.168.1.1) successfully, but traceroutes to the internet don't go
anywhere (the WRT54G is the default gateway since the ADSL modem is on
the other side). I have to do "route flush -inet" to get things going
again. Sometimes even that does not work and I have to reboot the
OpenBSD machine.

A segment from /var/log/messages shows this:

Feb 8 18:17:35 bytor /bsd: arplookup: unable to enter address for
192.168.1.1
Feb 8 18:17:36 bytor /bsd: arpresolve: can't allocate llinfo
Feb 8 18:17:41 bytor last message repeated 44 times
Feb 8 18:17:43 bytor routed[24809]: interface ral0 to 192.168.1.1 restored
Feb 8 18:17:44 bytor routed[24809]: punt RTM_ADD without gateway
Feb 8 18:17:45 bytor /bsd: arpresolve: can't allocate llinfo
Feb 8 18:17:49 bytor last message repeated 19 times
Feb 8 18:17:50 bytor routed[24809]: punt RTM_ADD without gateway
Feb 8 18:17:53 bytor /bsd: arpresolve: can't allocate llinfo
Feb 8 18:18:01 bytor last message repeated 7 times
Feb 8 18:18:03 bytor routed[24809]: punt RTM_ADD without gateway
Feb 8 18:18:13 bytor /bsd: arpresolve: can't allocate llinfo
Feb 8 18:18:44 bytor last message repeated 10 times
Feb 8 18:20:45 bytor last message repeated 86 times
Feb 8 18:20:45 bytor /bsd: arpresolve: can't allocate llinfo
Feb 8 18:20:47 bytor routed[24809]: interface ral0 to 192.168.1.2 bad:
in=4 ierr=4 out=2 oerr=0

If I do "arp -a" or "route -n show -inet" it shows nothing different
from when things are working perfectly and no error are being logged to
messages. I set an xterm going with "tcpdump -i ral0 arp" to check for
anything there, no luck. Just the same stiff I see when everything is
working:

19:49:41.403910 arp who-has 192.168.1.2 tell 192.168.1.1
19:49:41.403982 arp reply 192.168.1.2 is-at 0:12:17:a0:9a:81

192.168.1.1 is the WRT54G and it seems to ask this every 30s.
192.168.1.2 is the OpenBSD machine.

If somebody could help me with this, I would appreciate it. It's turning
into a real PITA.

Thanks in advance for any help.

[Otto Moerbeek]

Post by Cory Albrecht
Hello all,
This is becoming rather frustrating, and I'm hoping somebody can help me.
Every so often my OpenBSD stops routing packets and dmesg fills up with
"arpresolve: can't allocate llinfo". Before I go any further, let me
illustrate my network setup.
'Net---ADSL---WRT54G . . . (ral0)OpenBSD(em1)---Linux
. (em0)
. |
LapTop |
WinXP
Dashes are wired, dots are wireless.
Seemingly randomly, the OpenBSD just looses the ability to facilitate new IPv4
connections via ral0. While IPv4 connectons from my laptop fail (like share
mounted form teh Linux Samba server), IPv6 ssh sessions still work. If I try
to ping the IP associated with ral0 (192.168.1.2) I get no response.
If I go the the console of the OpenBSD machine, I can ping the WRT54G
(192.168.1.1) successfully, but traceroutes to the internet don't go anywhere
(the WRT54G is the default gateway since the ADSL modem is on the other side).
I have to do "route flush -inet" to get things going again. Sometimes even
that does not work and I have to reboot the OpenBSD machine.
Feb 8 18:17:35 bytor /bsd: arplookup: unable to enter address for 192.168.1.1
Feb 8 18:17:36 bytor /bsd: arpresolve: can't allocate llinfo
Feb 8 18:17:41 bytor last message repeated 44 times
Feb 8 18:17:43 bytor routed[24809]: interface ral0 to 192.168.1.1 restored
Feb 8 18:17:44 bytor routed[24809]: punt RTM_ADD without gateway
Feb 8 18:17:45 bytor /bsd: arpresolve: can't allocate llinfo
Feb 8 18:17:49 bytor last message repeated 19 times
Feb 8 18:17:50 bytor routed[24809]: punt RTM_ADD without gateway
Feb 8 18:17:53 bytor /bsd: arpresolve: can't allocate llinfo
Feb 8 18:18:01 bytor last message repeated 7 times
Feb 8 18:18:03 bytor routed[24809]: punt RTM_ADD without gateway
Feb 8 18:18:13 bytor /bsd: arpresolve: can't allocate llinfo
Feb 8 18:18:44 bytor last message repeated 10 times
Feb 8 18:20:45 bytor last message repeated 86 times
Feb 8 18:20:45 bytor /bsd: arpresolve: can't allocate llinfo
Feb 8 18:20:47 bytor routed[24809]: interface ral0 to 192.168.1.2 bad: in=4
ierr=4 out=2 oerr=0
If I do "arp -a" or "route -n show -inet" it shows nothing different from when
things are working perfectly and no error are being logged to messages. I set
an xterm going with "tcpdump -i ral0 arp" to check for anything there, no
19:49:41.403910 arp who-has 192.168.1.2 tell 192.168.1.1
19:49:41.403982 arp reply 192.168.1.2 is-at 0:12:17:a0:9a:81
192.168.1.1 is the WRT54G and it seems to ask this every 30s. 192.168.1.2 is
the OpenBSD machine.
If somebody could help me with this, I would appreciate it. It's turning into
a real PITA.
Thanks in advance for any help.

Show the interface configs. Very likely you have two interfaces that
fall into the same subnet.

-Otto

[Otto Moerbeek]

[snip]

Post by Otto Moerbeek

Post by Cory Albrecht
Thanks in advance for any help.

Show the interface configs. Very likely you have two interfaces that
fall into the same subnet.

I tried to reply to your private reply, but your receiving mailer uses
sorbs. Since my DSL address is using listed as "dynamic ip" (while it
is static), your MX does not accept my mails. And yes, I tried to get
it delisted from sorbs, without success. Sorbs just sucks.

So here's my repl again:

ok, that looks good indeed. Next question: why are you runing routed?
That might make the trouble shooting harder. Also, you could try the
diff posted, to get more detailed info.

-Otto

[Darren Tucker]

Post by Cory Albrecht
Feb 8 18:17:36 bytor /bsd: arpresolve: can't allocate llinfo

I had that at one point. I added this to the kernel to figure out what
it was complaining about so I could fix it. I then promptly forgot
what the cause was, but I've still got the diff in my local source tree
and it might be of some help to you.

Disclaimer: I normally stay out of sys/ so YMMV.

Index: netinet/if_ether.c
===================================================================
RCS file: /cvs/src/sys/netinet/if_ether.c,v
retrieving revision 1.65
diff -u -p -r1.65 if_ether.c
--- netinet/if_ether.c 21 Aug 2006 21:36:53 -0000 1.65
+++ netinet/if_ether.c 11 Feb 2007 10:43:51 -0000
@@ -392,7 +392,10 @@ arpresolve(ac, rt, m, dst, desten)
rt = la->la_rt;
}
if (la == 0 || rt == 0) {
- log(LOG_DEBUG, "arpresolve: can't allocate llinfo\n");
+ log(LOG_DEBUG, "arpresolve: can't allocate llinfo for "
+ "%s:%s%s\n", inet_ntoa(SIN(dst)->sin_addr),
+ la == NULL ? " no link address" : "",
+ rt == NULL ? " no route" : "");
m_freem(m);
return (0);
}

[Berk D. Demir]

Post by Darren Tucker
Index: netinet/if_ether.c
===================================================================
RCS file: /cvs/src/sys/netinet/if_ether.c,v
retrieving revision 1.65
diff -u -p -r1.65 if_ether.c
--- netinet/if_ether.c 21 Aug 2006 21:36:53 -0000 1.65
+++ netinet/if_ether.c 11 Feb 2007 10:43:51 -0000
@@ -392,7 +392,10 @@ arpresolve(ac, rt, m, dst, desten)
rt = la->la_rt;
}
if (la == 0 || rt == 0) {
- log(LOG_DEBUG, "arpresolve: can't allocate llinfo\n");
+ log(LOG_DEBUG, "arpresolve: can't allocate llinfo for "
+ "%s:%s%s\n", inet_ntoa(SIN(dst)->sin_addr),
+ la == NULL ? " no link address" : "",
+ rt == NULL ? " no route" : "");
m_freem(m);
return (0);
}

Oh yes. Thumbs up for this patch.
Have a very similar one in my tree for months.

[Cory Albrecht]

Hello all,

My OpenBSD firewall is still randomly stopping routing packets and I
still can't figure out why. :-(

I made the suggested patch to if_ether.c, ut now I just get the
following line in /var log messages:

Feb 14 18:08:41 bytor /bsd: arpresolve: can't allocate llinfo for
192.168.1.1:no link address


Symptoms: Firewall can ping the wifi router (to which ADSL modem is
attached), but pinging anything beyond it fails. If I try to traceroute
to some place beyond the router, it doesn't show the router as the first
hop. (If it can ping the router, shouldn't it show up a the first hop on
a traceroute?). Even though the firewall can ping the router, it cannot
ping my laptop, even though the route to both goes out ral0. The laptop
cannot ping the firewall either. I know the router is still working
because my laptop can still access the internet through it once I reset
the default gateway to the router instead of the firewall. IPv6 ssh
connections form the laptop to the firewall stay active.

Things is, "arp -a" and "route -n show -inet" show extactly the same
thing whether the problem is currently in progress or everything is
working perfectly. No NICs accidentally have addresses on the wrong segment.

I had routed running, but stopping it has made no difference.

Anybody have any ideas?

[***@bytor] 1:03:58 [9]/etc> arp -a
bytor (192.168.0.1) at 00:0e:0c:bc:38:9d on em1 static
xanadu (192.168.0.2) at 00:0e:0c:b9:4d:ed on em1
heechee.wireless (192.168.1.1) at 00:13:10:0e:0b:08 on ral0
snowdog.wireless (192.168.1.3) at 00:12:17:60:fe:40 on ral0
redbarchetta.wireless.fenris.cjb.net (192.168.1.191) at
00:18:de:20:4f:2e on ral0
bytor (192.168.16.1) at 00:0e:0c:b9:50:74 on em0 static
snowdog (192.168.16.2) at 00:15:f2:e8:7f:51 on em0

[***@bytor] 1:04:03 [10]/etc> route -n show -inet
Routing tables

Internet:
Destination Gateway Flags Refs Use Mtu Interface
default 192.168.1.1 UGS 16 188916 - ral0
127.0.0.1 127.0.0.1 UH 2 6049 33224 lo0
192.168.0/24 link#3 UC 2 0 - em1
192.168.0.1 00:0e:0c:bc:38:9d UHLc 9 996889 - lo0
192.168.0.2 00:0e:0c:b9:4d:ed UHLc 1 56064 - em1
192.168.1/24 link#4 UC 3 0 - ral0
192.168.1.1 00:13:10:0e:0b:08 UHLc 2 3272 - ral0
192.168.1.3 00:12:17:60:fe:40 UHLc 0 483 - ral0
192.168.1.191 00:18:de:20:4f:2e UHLc 0 4587 - ral0
192.168.2/24 link#1 UC 0 0 - fxp0
192.168.16/24 link#2 UC 2 0 - em0
192.168.16.1 00:0e:0c:b9:50:74 UHLc 0 50 - lo0
192.168.16.2 00:15:f2:e8:7f:51 UHLc 5 392664 - em0

[***@bytor] 1:04:13 [11]/etc> cat hostname.ral0
inet 192.168.1.2 255.255.255.0 192.168.1.255 nwid fenris nwkey
0x0A18135EB54723927B64AB65BC
inet6 alias 2001:05c0:92cf:1::c0a8:0102 64

[***@bytor] 1:06:08 [12]/etc> cat hostname.em0
inet 192.168.16.1 255.255.255.0 192.168.16.255
inet6 alias 2001:05c0:92cf:10::c0a8:1001 64

[***@bytor] 1:06:18 [13]/etc> cat hostname.em1
inet 192.168.0.1 255.255.255.0 192.168.0.255
inet6 alias 2001:05c0:92cf:0::c0a8:0001 64

[***@bytor] 1:06:33 [14]/etc> cat hostname.fxp0
inet 192.168.2.1 255.255.255.0 192.168.2.255
inet6 alias 2001:5c0:92cf:2::c0a8:0201 64

[Shane Lahey]

Hello Cory,

Post by Cory Albrecht
Hello all,
My OpenBSD firewall is still randomly stopping routing packets and I
still can't figure out why. :-(
I made the suggested patch to if_ether.c, ut now I just get the
Feb 14 18:08:41 bytor /bsd: arpresolve: can't allocate llinfo for
192.168.1.1:no link address
Symptoms: Firewall can ping the wifi router (to which ADSL modem is
attached), but pinging anything beyond it fails. If I try to traceroute
to some place beyond the router, it doesn't show the router as the first
hop. (If it can ping the router, shouldn't it show up a the first hop on
a traceroute?). Even though the firewall can ping the router, it cannot
ping my laptop, even though the route to both goes out ral0. The laptop
cannot ping the firewall either. I know the router is still working
because my laptop can still access the internet through it once I reset
the default gateway to the router instead of the firewall. IPv6 ssh
connections form the laptop to the firewall stay active.
Things is, "arp -a" and "route -n show -inet" show extactly the same
thing whether the problem is currently in progress or everything is
working perfectly. No NICs accidentally have addresses on the wrong segment.
I had routed running, but stopping it has made no difference.
Anybody have any ideas?
bytor (192.168.0.1) at 00:0e:0c:bc:38:9d on em1 static
xanadu (192.168.0.2) at 00:0e:0c:b9:4d:ed on em1
heechee.wireless (192.168.1.1) at 00:13:10:0e:0b:08 on ral0
snowdog.wireless (192.168.1.3) at 00:12:17:60:fe:40 on ral0
redbarchetta.wireless.fenris.cjb.net (192.168.1.191) at
00:18:de:20:4f:2e on ral0
bytor (192.168.16.1) at 00:0e:0c:b9:50:74 on em0 static
snowdog (192.168.16.2) at 00:15:f2:e8:7f:51 on em0
Routing tables
Destination Gateway Flags Refs Use Mtu Interface
default 192.168.1.1 UGS 16 188916 - ral0
127.0.0.1 127.0.0.1 UH 2 6049 33224 lo0
192.168.0/24 link#3 UC 2 0 - em1
192.168.0.1 00:0e:0c:bc:38:9d UHLc 9 996889 - lo0
192.168.0.2 00:0e:0c:b9:4d:ed UHLc 1 56064 - em1
192.168.1/24 link#4 UC 3 0 - ral0
192.168.1.1 00:13:10:0e:0b:08 UHLc 2 3272 - ral0
192.168.1.3 00:12:17:60:fe:40 UHLc 0 483 - ral0
192.168.1.191 00:18:de:20:4f:2e UHLc 0 4587 - ral0
192.168.2/24 link#1 UC 0 0 - fxp0
192.168.16/24 link#2 UC 2 0 - em0
192.168.16.1 00:0e:0c:b9:50:74 UHLc 0 50 - lo0
192.168.16.2 00:15:f2:e8:7f:51 UHLc 5 392664 - em0
inet 192.168.1.2 255.255.255.0 192.168.1.255 nwid fenris nwkey
0x0A18135EB54723927B64AB65BC
inet6 alias 2001:05c0:92cf:1::c0a8:0102 64
inet 192.168.16.1 255.255.255.0 192.168.16.255
inet6 alias 2001:05c0:92cf:10::c0a8:1001 64
inet 192.168.0.1 255.255.255.0 192.168.0.255
inet6 alias 2001:05c0:92cf:0::c0a8:0001 64
inet 192.168.2.1 255.255.255.0 192.168.2.255
inet6 alias 2001:5c0:92cf:2::c0a8:0201 64

I had this issue before and it turned out to be a bad NIC.