Discussion:
OSPF over gif on top of IPsec transport -current
(too old to reply)
Atanas Vladimirov
2018-03-04 11:08:21 UTC
Permalink
Raw Message
Hi,

I can't make OSPF to work on gif over IPsec.
With tcpdump on gif I see the OSPFv2-hello only from localhost:

# R1
[ns]~$ tcpdump -nei gif0
tcpdump: listening on gif0, link-type LOOP
23:19:29.181685 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid
192.168.1.1 area 0.0.0.1 [tos 0xc0] [ttl 1]
23:19:39.192025 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid
192.168.1.1 area 0.0.0.1 [tos 0xc0] [ttl 1]
23:19:49.202372 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid
192.168.1.1 area 0.0.0.1 [tos 0xc0] [ttl 1]
23:19:59.212730 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid
192.168.1.1 area 0.0.0.1 [tos 0xc0] [ttl 1]
23:20:09.223064 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid
192.168.1.1 area 0.0.0.1 [tos 0xc0] [ttl 1]
23:20:19.233393 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid
192.168.1.1 area 0.0.0.1 [tos 0xc0] [ttl 1]

# R2
[hodor]~$ tcpdump -nei gif0
tcpdump: listening on gif0, link-type LOOP
12:51:59.316704 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid
172.16.1.1 backbone [tos 0xc0] [ttl 1]
12:52:09.327002 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid
172.16.1.1 backbone [tos 0xc0] [ttl 1]
12:52:19.337314 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid
172.16.1.1 backbone [tos 0xc0] [ttl 1]

While on enc0 both hello's appears (not sure if `bad ip cksum` is the
reason for my issues):

# R1
[ns]~$ tcpdump -nvi enc0
tcpdump: listening on enc0, link-type ENC
12:24:37.625873 (authentic,confidential): SPI 0x11af4dae: 93.123.39.67 >
95.87.227.232: 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid
172.16.1.1 backbone E mask 255.255.255.255 int 10 pri 1 dead 40 nbrs
[tos 0xc0] [ttl 1] (id 25841, len 64) (ttl 60, id 37752, len 84)
12:24:41.882173 (authentic,confidential): SPI 0x1a3fbc6d: 95.87.227.232
93.123.39.67: 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid 192.168.1.1 backbone E mask 255.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1] (id 27818, len 64) (ttl 64, id 60563, len 84, bad ip cksum 32d7! -> c614)
12:24:47.636188 (authentic,confidential): SPI 0x11af4dae: 93.123.39.67 >
95.87.227.232: 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid
172.16.1.1 backbone E mask 255.255.255.255 int 10 pri 1 dead 40 nbrs
[tos 0xc0] [ttl 1] (id 36067, len 64) (ttl 60, id 65348, len 84)
12:24:51.892467 (authentic,confidential): SPI 0x1a3fbc6d: 95.87.227.232
93.123.39.67: 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid 192.168.1.1 backbone E mask 255.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1] (id 5127, len 64) (ttl 64, id 12476, len 84, bad ip cksum 201! -> 81ec)
12:24:57.646535 (authentic,confidential): SPI 0x11af4dae: 93.123.39.67 >
95.87.227.232: 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid
172.16.1.1 backbone E mask 255.255.255.255 int 10 pri 1 dead 40 nbrs
[tos 0xc0] [ttl 1] (id 39220, len 64) (ttl 60, id 1938, len 84)

# R2
[hodor]~$ tcpdump -nvi enc0 | grep OSPF
tcpdump: listening on enc0, link-type ENC
12:28:57.894007 (authentic,confidential): SPI 0x11af4dae: 93.123.39.67 >
95.87.227.232: 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid
172.16.1.1 backbone E mask 255.255.255.255 int 10 pri 1 dead 40 nbrs
[tos 0xc0] [ttl 1] (id 3667, len 64) (ttl 64, id 14037, len 84, bad ip
cksum 2b6d! -> 7bd3)
12:29:02.151763 (authentic,confidential): SPI 0x1a3fbc6d: 95.87.227.232
93.123.39.67: 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid 192.168.1.1 backbone E mask 25
5.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1] (id 16974,
len 64) (ttl 60, id 21648, len 84)
12:29:07.904315 (authentic,confidential): SPI 0x11af4dae: 93.123.39.67 >
95.87.227.232: 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid
172.16.1.1 backbone E mask 255
.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1] (id 45590, len
64) (ttl 64, id 35262, len 84, bad ip cksum 2743! -> 28ea)
12:29:12.162049 (authentic,confidential): SPI 0x1a3fbc6d: 95.87.227.232
93.123.39.67: 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid 192.168.1.1 backbone E mask 25
5.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1] (id 19966,
len 64) (ttl 60, id 3134, len 84)
12:29:17.914621 (authentic,confidential): SPI 0x11af4dae: 93.123.39.67 >
95.87.227.232: 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid
172.16.1.1 backbone E mask 255
.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1] (id 36161, len
64) (ttl 64, id 53105, len 84, bad ip cksum fcb8! -> e336)
12:29:22.172468 (authentic,confidential): SPI 0x1a3fbc6d: 95.87.227.232
93.123.39.67: 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid 192.168.1.1 backbone E mask 25
5.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1] (id 36221,
len 64) (ttl 60, id 29514, len 84)

If I set a static routes the regular traffic flows as it should.

The configs are the same on both routers:

# R1
[ns]~$ doas cat /etc/ipsec.conf
local_ip="95.87.227.232"
remote_ip="93.123.39.67"
ike esp transport from $local_ip to $remote_ip

# R2
[hodor]~$ doas cat /etc/ipsec.conf
local_ip="93.123.39.67"
remote_ip="95.87.227.232"
ike esp transport from $local_ip to $remote_ip

# R1
[ns]~$ doas cat /etc/hostname.gif0
up
mtu 1400
tunnel 95.87.227.232 93.123.39.67
inet 10.255.255.2/32
dest 10.255.255.1

# R2
[hodor]~$ doas cat /etc/hostname.gif0
up
mtu 1400
tunnel 93.123.39.67 95.87.227.232
inet 10.255.255.1/32
dest 10.255.255.2

# R1
[ns]~$ doas cat /etc/ospfd.conf
# $OpenBSD: ospfd.conf,v 1.1 2014/07/11 16:36:35 deraadt Exp $

# macros
password="secret"

# global configuration
router-id 192.168.1.1
fib-update yes
redistribute connected

# areas
area 0.0.0.0 {
interface gif0
}

# R2
[hodor]~$ doas cat /etc/ospfd.conf
# $OpenBSD: ospfd.conf,v 1.1 2014/07/11 16:36:35 deraadt Exp $

# macros
password="secret"

# global configuration
router-id 172.16.1.1
fib-update yes
redistribute connected

# areas
area 0.0.0.0 {
interface vether0
interface gif0
}

# R1
[ns]~$ ospfctl sh nei
ID Pri State DeadTime Address Iface
Uptime

# R1
[ns]~$ ospfctl sh int
Interface Address State HelloTimer Linkstate Uptime nc
ac
gif0 10.255.255.2/32 P2P 00:00:04 unknown 00:15:37 0
0

# R2
[hodor]~$ ospfctl sh nei
ID Pri State DeadTime Address Iface
Uptime
172.16.1.9 1 FULL/DR 00:00:36 172.16.1.9 vether0
02:10:14

# R2
[hodor]~$ ospfctl sh int
Interface Address State HelloTimer Linkstate Uptime nc
ac
gif0 10.255.255.1/32 P2P 00:00:04 unknown 02:10:24 0
0
vether0 172.16.1.1/24 BCKUP 00:00:09 active 02:10:24 1
1

Please, let me know if I'm doing something wrong/stupid or this is bug
somewhere in the stack.
As a side note - eigrpd behaves the same way.

# dmesg

dmesg:
OpenBSD 6.3-beta (GENERIC.MP) #26: Fri Mar 2 22:56:04 MST 2018
***@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 17132367872 (16338MB)
avail mem = 16606117888 (15836MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xeb5a0 (56 entries)
bios0: vendor American Megatrends Inc. version "2.2" date 02/20/2015
bios0: Thomas-Krenn.AG X9SCL/X9SCM
acpi0 at bios0: rev 2
acpi0: sleep states S0 S1 S4 S5
acpi0: tables DSDT FACP APIC FPDT MCFG HPET SSDT SPMI SSDT SSDT DMAR
acpi0: wakeup devices PS2K(S4) UAR1(S4) UAR2(S4) P0P1(S4) USB1(S4)
USB2(S4) USB3(S4) USB4(S4) USB5(S4) USB6(S4) USB7(S4) PXSX(S4) RP01(S4)
PXSX(S4) RP02(S4) PXSX(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz, 3300.49 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT,MELTDOWN
cpu0: 256KB 64b/line 8-way L2 cache
acpitimer0: recalibrated TSC frequency 3300030874 Hz
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 100MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz, 3300.02 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT,MELTDOWN
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz, 3300.02 MHz
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT,MELTDOWN
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 6 (application processor)
cpu3: Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz, 3300.02 MHz
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT,MELTDOWN
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 0, core 3, package 0
cpu4 at mainbus0: apid 1 (application processor)
cpu4: Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz, 3300.03 MHz
cpu4:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT,MELTDOWN
cpu4: 256KB 64b/line 8-way L2 cache
cpu4: smt 1, core 0, package 0
cpu5 at mainbus0: apid 3 (application processor)
cpu5: Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz, 3300.03 MHz
cpu5:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT,MELTDOWN
cpu5: 256KB 64b/line 8-way L2 cache
cpu5: smt 1, core 1, package 0
cpu6 at mainbus0: apid 5 (application processor)
cpu6: Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz, 3300.02 MHz
cpu6:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT,MELTDOWN
cpu6: 256KB 64b/line 8-way L2 cache
cpu6: smt 1, core 2, package 0
cpu7 at mainbus0: apid 7 (application processor)
cpu7: Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz, 3300.02 MHz
cpu7:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT,MELTDOWN
cpu7: 256KB 64b/line 8-way L2 cache
cpu7: smt 1, core 3, package 0
ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 20, 24 pins
acpimcfg0 at acpi0 addr 0xe0000000, bus 0-255
acpihpet0 at acpi0: 14318179 Hz
acpihpet0: recalibrated TSC frequency 3300008959 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 7 (P0P1)
acpiprt2 at acpi0: bus 5 (RP01)
acpiprt3 at acpi0: bus -1 (RP02)
acpiprt4 at acpi0: bus -1 (RP03)
acpiprt5 at acpi0: bus -1 (RP04)
acpiprt6 at acpi0: bus 6 (RP05)
acpiprt7 at acpi0: bus -1 (RP06)
acpiprt8 at acpi0: bus -1 (RP07)
acpiprt9 at acpi0: bus -1 (RP08)
acpiprt10 at acpi0: bus 1 (PEG0)
acpiprt11 at acpi0: bus -1 (PEG1)
acpiprt12 at acpi0: bus -1 (PEG2)
acpiprt13 at acpi0: bus -1 (PEG3)
acpiec0 at acpi0: not present
acpicpu0 at acpi0: C3(***@80 ***@0x20), C1(***@1 mwait.1), PSS
acpicpu1 at acpi0: C3(***@80 ***@0x20), C1(***@1 mwait.1), PSS
acpicpu2 at acpi0: C3(***@80 ***@0x20), C1(***@1 mwait.1), PSS
acpicpu3 at acpi0: C3(***@80 ***@0x20), C1(***@1 mwait.1), PSS
acpicpu4 at acpi0: C3(***@80 ***@0x20), C1(***@1 mwait.1), PSS
acpicpu5 at acpi0: C3(***@80 ***@0x20), C1(***@1 mwait.1), PSS
acpicpu6 at acpi0: C3(***@80 ***@0x20), C1(***@1 mwait.1), PSS
acpicpu7 at acpi0: C3(***@80 ***@0x20), C1(***@1 mwait.1), PSS
acpipwrres0 at acpi0: FN00, resource for FAN0
acpipwrres1 at acpi0: FN01, resource for FAN1
acpipwrres2 at acpi0: FN02, resource for FAN2
acpipwrres3 at acpi0: FN03, resource for FAN3
acpipwrres4 at acpi0: FN04, resource for FAN4
acpitz0 at acpi0: critical temperature is 106 degC
acpitz1 at acpi0: critical temperature is 106 degC
"INT3F0D" at acpi0 not configured
"IPI0001" at acpi0 not configured
acpibtn0 at acpi0: PWRB
"PNP0C0B" at acpi0 not configured
"PNP0C0B" at acpi0 not configured
"PNP0C0B" at acpi0 not configured
"PNP0C0B" at acpi0 not configured
"PNP0C0B" at acpi0 not configured
acpivideo0 at acpi0: GFX0
acpivout0 at acpivideo0: DD01
acpivout at acpivideo0 not configured
acpivout at acpivideo0 not configured
acpivout at acpivideo0 not configured
acpivout at acpivideo0 not configured
acpivout at acpivideo0 not configured
acpivout at acpivideo0 not configured
acpivout at acpivideo0 not configured
ipmi at mainbus0 not configured
cpu0: Enhanced SpeedStep 3300 MHz: speeds: 3301, 3300, 3200, 3100, 2900,
2800, 2700, 2600, 2400, 2300, 2200, 2100, 2000, 1800, 1700, 1600 MHz
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel Xeon E3-1200 v2 Host" rev 0x09
ppb0 at pci0 dev 1 function 0 "Intel Core 3G PCIE" rev 0x09: msi
pci1 at ppb0 bus 1
ppb1 at pci1 dev 0 function 0 "IDT 89HPES12N3A" rev 0x0e
pci2 at ppb1 bus 2
ppb2 at pci2 dev 2 function 0 "IDT 89HPES12N3A" rev 0x0e
pci3 at ppb2 bus 3
em0 at pci3 dev 0 function 0 "Intel 82571EB" rev 0x06: apic 2 int 19,
address 00:15:17:bc:a9:65
em1 at pci3 dev 0 function 1 "Intel 82571EB" rev 0x06: apic 2 int 18,
address 00:15:17:bc:a9:64
ppb3 at pci2 dev 4 function 0 "IDT 89HPES12N3A" rev 0x0e
pci4 at ppb3 bus 4
em2 at pci4 dev 0 function 0 "Intel 82571EB" rev 0x06: apic 2 int 17,
address 00:15:17:bc:a9:67
em3 at pci4 dev 0 function 1 "Intel 82571EB" rev 0x06: apic 2 int 16,
address 00:15:17:bc:a9:66
em4 at pci0 dev 25 function 0 "Intel 82579LM" rev 0x05: msi, address
00:25:90:a1:c0:19
ehci0 at pci0 dev 26 function 0 "Intel 6 Series USB" rev 0x05: apic 2
int 16
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 configuration 1 interface 0 "Intel EHCI root hub" rev
2.00/1.00 addr 1
ppb4 at pci0 dev 28 function 0 "Intel 6 Series PCIE" rev 0xb5: msi
pci5 at ppb4 bus 5
ppb5 at pci0 dev 28 function 4 "Intel 6 Series PCIE" rev 0xb5: msi
pci6 at ppb5 bus 6
em5 at pci6 dev 0 function 0 "Intel 82574L" rev 0x00: msi, address
00:25:90:a1:c0:18
ehci1 at pci0 dev 29 function 0 "Intel 6 Series USB" rev 0x05: apic 2
int 23
usb1 at ehci1: USB revision 2.0
uhub1 at usb1 configuration 1 interface 0 "Intel EHCI root hub" rev
2.00/1.00 addr 1
ppb6 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0xa5
pci7 at ppb6 bus 7
vga1 at pci7 dev 3 function 0 "Matrox MGA G200eW" rev 0x0a
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
pcib0 at pci0 dev 31 function 0 "Intel C204 LPC" rev 0x05
ahci0 at pci0 dev 31 function 2 "Intel 6 Series AHCI" rev 0x05: msi,
AHCI 1.3
ahci0: port 0: 3.0Gb/s
ahci0: port 2: 3.0Gb/s
scsibus1 at ahci0: 32 targets
sd0 at scsibus1 targ 0 lun 0: <ATA, WDC WD6401AALS-0, 05.0> SCSI3
0/direct fixed naa.50014ee002288208
sd0: 610480MB, 512 bytes/sector, 1250263728 sectors
sd1 at scsibus1 targ 2 lun 0: <ATA, WDC WD6400AAKS-7, 01.0> SCSI3
0/direct fixed naa.50014ee2adc0c1df
sd1: 610480MB, 512 bytes/sector, 1250263728 sectors
ichiic0 at pci0 dev 31 function 3 "Intel 6 Series SMBus" rev 0x05: apic
2 int 18
iic0 at ichiic0
sdtemp0 at iic0 addr 0x19: mcp98243
sdtemp1 at iic0 addr 0x1b: mcp98243
spdmem0 at iic0 addr 0x51: 8GB DDR3 SDRAM ECC PC3-12800 with thermal
sensor
spdmem1 at iic0 addr 0x53: 8GB DDR3 SDRAM ECC PC3-12800 with thermal
sensor
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
com2 at isa0 port 0x3e8/8 irq 5: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pckbd0 at pckbc0 (kbd slot)
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
wbsio0 at isa0 port 0x2e/2: NCT6776F rev 0x33
lm1 at wbsio0 port 0xa30/8: NCT6776F
vmm0 at mainbus0: VMX/EPT
uhub2 at uhub0 port 1 configuration 1 interface 0 "Intel Rate Matching
Hub" rev 2.00/0.00 addr 2
uhidev0 at uhub2 port 2 configuration 1 interface 0 "Winbond Electronics
Corp Hermon USB hidmouse Device" rev 1.10/0.01 addr 3
uhidev0: iclass 3/1
ums0 at uhidev0: 3 buttons, Z dir
wsmouse0 at ums0 mux 0
uhidev1 at uhub2 port 2 configuration 1 interface 1 "Winbond Electronics
Corp Hermon USB hidmouse Device" rev 1.10/0.01 addr 3
uhidev1: iclass 3/1
ukbd0 at uhidev1: 8 variable keys, 6 key codes
wskbd1 at ukbd0 mux 1
wskbd1: connecting to wsdisplay0
uhub3 at uhub1 port 1 configuration 1 interface 0 "Intel Rate Matching
Hub" rev 2.00/0.00 addr 2
uftdi0 at uhub3 port 3 configuration 1 interface 0 "FTDI FT232R USB
UART" rev 2.00/6.00 addr 3
ucom0 at uftdi0 portno 1
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
sd2 at scsibus3 targ 1 lun 0: <OPENBSD, SR RAID 1, 006> SCSI2 0/direct
fixed
sd2: 610477MB, 512 bytes/sector, 1250258033 sectors
root on sd2a (391770920b7e6ed3.a) swap on sd2b dump on sd2b
Stefan Sperling
2018-03-04 11:31:17 UTC
Permalink
Raw Message
Post by Atanas Vladimirov
Please, let me know if I'm doing something wrong/stupid or this is bug
somewhere in the stack.
I can't spot anything wrong in what you've shown but it seems you're
not looking at all the data you could be looking at.

What might help with diagnosing the issue is monitoring the output of:

netstat -I gif0
netstat -I enc0

and:
netstat -s

Look closely at how the counters change, and find the ones which
could relate to an OSPF packet being dropped.

Also, check if pf is dropping related packets by logging any blocking
rules and checking pflog0 with tcpdump as well.
Atanas Vladimirov
2018-03-04 13:40:12 UTC
Permalink
Raw Message
Post by Stefan Sperling
Post by Atanas Vladimirov
Please, let me know if I'm doing something wrong/stupid or this is bug
somewhere in the stack.
I can't spot anything wrong in what you've shown but it seems you're
not looking at all the data you could be looking at.
netstat -I gif0
netstat -I enc0
netstat -s
Look closely at how the counters change, and find the ones which
could relate to an OSPF packet being dropped.
Also, check if pf is dropping related packets by logging any blocking
rules and checking pflog0 with tcpdump as well.
Hi Stefan,

I forgot to mention that both gif0 and enc0 are disable in pf.conf (set
skip on {...}).
Also I have a `pass quick log proto ospf` rule.

With `netstat` I observe the same behavior, packets going out on gif0 -
no packets in.

ns]~$ netstat -I gif0
Name Mtu Network Address Ipkts Ierrs Opkts
Oerrs Colls
gif0 1400 <Link> 0 0 8142
0 0
gif0 1400 10.255.255. 10.255.255.2 0 0 8142
0 0
[ns]~$ netstat -I enc0
Name Mtu Network Address Ipkts Ierrs Opkts
Oerrs Colls
enc0* 0 <Link> 8820 0 8870
0 0

I'll try to take a deeper look on this.
Thanks for your time and effort,
Atanas
Remi Locherer
2018-03-09 17:13:10 UTC
Permalink
Raw Message
Post by Atanas Vladimirov
Hi,
I can't make OSPF to work on gif over IPsec.
# R1
[ns]~$ tcpdump -nei gif0
tcpdump: listening on gif0, link-type LOOP
23:19:29.181685 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid
192.168.1.1 area 0.0.0.1 [tos 0xc0] [ttl 1]
23:19:39.192025 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid
192.168.1.1 area 0.0.0.1 [tos 0xc0] [ttl 1]
23:19:49.202372 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid
192.168.1.1 area 0.0.0.1 [tos 0xc0] [ttl 1]
23:19:59.212730 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid
192.168.1.1 area 0.0.0.1 [tos 0xc0] [ttl 1]
23:20:09.223064 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid
192.168.1.1 area 0.0.0.1 [tos 0xc0] [ttl 1]
23:20:19.233393 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid
192.168.1.1 area 0.0.0.1 [tos 0xc0] [ttl 1]
# R2
[hodor]~$ tcpdump -nei gif0
tcpdump: listening on gif0, link-type LOOP
12:51:59.316704 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid 172.16.1.1
backbone [tos 0xc0] [ttl 1]
12:52:09.327002 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid 172.16.1.1
backbone [tos 0xc0] [ttl 1]
12:52:19.337314 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid 172.16.1.1
backbone [tos 0xc0] [ttl 1]
While on enc0 both hello's appears (not sure if `bad ip cksum` is the reason
# R1
[ns]~$ tcpdump -nvi enc0
tcpdump: listening on enc0, link-type ENC
12:24:37.625873 (authentic,confidential): SPI 0x11af4dae: 93.123.39.67 >
95.87.227.232: 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid 172.16.1.1
backbone E mask 255.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1]
(id 25841, len 64) (ttl 60, id 37752, len 84)
12:24:41.882173 (authentic,confidential): SPI 0x1a3fbc6d: 95.87.227.232 >
93.123.39.67: 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid 192.168.1.1
backbone E mask 255.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1]
(id 27818, len 64) (ttl 64, id 60563, len 84, bad ip cksum 32d7! -> c614)
12:24:47.636188 (authentic,confidential): SPI 0x11af4dae: 93.123.39.67 >
95.87.227.232: 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid 172.16.1.1
backbone E mask 255.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1]
(id 36067, len 64) (ttl 60, id 65348, len 84)
12:24:51.892467 (authentic,confidential): SPI 0x1a3fbc6d: 95.87.227.232 >
93.123.39.67: 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid 192.168.1.1
backbone E mask 255.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1]
(id 5127, len 64) (ttl 64, id 12476, len 84, bad ip cksum 201! -> 81ec)
12:24:57.646535 (authentic,confidential): SPI 0x11af4dae: 93.123.39.67 >
95.87.227.232: 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid 172.16.1.1
backbone E mask 255.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1]
(id 39220, len 64) (ttl 60, id 1938, len 84)
# R2
[hodor]~$ tcpdump -nvi enc0 | grep OSPF
tcpdump: listening on enc0, link-type ENC
12:28:57.894007 (authentic,confidential): SPI 0x11af4dae: 93.123.39.67 >
95.87.227.232: 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid 172.16.1.1
backbone E mask 255.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1]
(id 3667, len 64) (ttl 64, id 14037, len 84, bad ip cksum 2b6d! -> 7bd3)
12:29:02.151763 (authentic,confidential): SPI 0x1a3fbc6d: 95.87.227.232 >
93.123.39.67: 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid 192.168.1.1
backbone E mask 25
5.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1] (id 16974, len
64) (ttl 60, id 21648, len 84)
12:29:07.904315 (authentic,confidential): SPI 0x11af4dae: 93.123.39.67 >
95.87.227.232: 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid 172.16.1.1
backbone E mask 255
.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1] (id 45590, len 64)
(ttl 64, id 35262, len 84, bad ip cksum 2743! -> 28ea)
12:29:12.162049 (authentic,confidential): SPI 0x1a3fbc6d: 95.87.227.232 >
93.123.39.67: 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid 192.168.1.1
backbone E mask 25
5.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1] (id 19966, len
64) (ttl 60, id 3134, len 84)
12:29:17.914621 (authentic,confidential): SPI 0x11af4dae: 93.123.39.67 >
95.87.227.232: 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid 172.16.1.1
backbone E mask 255
.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1] (id 36161, len 64)
(ttl 64, id 53105, len 84, bad ip cksum fcb8! -> e336)
12:29:22.172468 (authentic,confidential): SPI 0x1a3fbc6d: 95.87.227.232 >
93.123.39.67: 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid 192.168.1.1
backbone E mask 25
5.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1] (id 36221, len
64) (ttl 60, id 29514, len 84)
If I set a static routes the regular traffic flows as it should.
# R1
[ns]~$ doas cat /etc/ipsec.conf
local_ip="95.87.227.232"
remote_ip="93.123.39.67"
ike esp transport from $local_ip to $remote_ip
# R2
[hodor]~$ doas cat /etc/ipsec.conf
local_ip="93.123.39.67"
remote_ip="95.87.227.232"
ike esp transport from $local_ip to $remote_ip
# R1
[ns]~$ doas cat /etc/hostname.gif0
up
mtu 1400
tunnel 95.87.227.232 93.123.39.67
inet 10.255.255.2/32
dest 10.255.255.1
# R2
[hodor]~$ doas cat /etc/hostname.gif0
up
mtu 1400
tunnel 93.123.39.67 95.87.227.232
inet 10.255.255.1/32
dest 10.255.255.2
# R1
[ns]~$ doas cat /etc/ospfd.conf
# $OpenBSD: ospfd.conf,v 1.1 2014/07/11 16:36:35 deraadt Exp $
# macros
password="secret"
# global configuration
router-id 192.168.1.1
fib-update yes
redistribute connected
# areas
area 0.0.0.0 {
interface gif0
}
# R2
[hodor]~$ doas cat /etc/ospfd.conf
# $OpenBSD: ospfd.conf,v 1.1 2014/07/11 16:36:35 deraadt Exp $
# macros
password="secret"
# global configuration
router-id 172.16.1.1
fib-update yes
redistribute connected
# areas
area 0.0.0.0 {
interface vether0
interface gif0
}
# R1
[ns]~$ ospfctl sh nei
ID Pri State DeadTime Address Iface Uptime
# R1
[ns]~$ ospfctl sh int
Interface Address State HelloTimer Linkstate Uptime nc ac
gif0 10.255.255.2/32 P2P 00:00:04 unknown 00:15:37 0 0
# R2
[hodor]~$ ospfctl sh nei
ID Pri State DeadTime Address Iface Uptime
172.16.1.9 1 FULL/DR 00:00:36 172.16.1.9 vether0 02:10:14
# R2
[hodor]~$ ospfctl sh int
Interface Address State HelloTimer Linkstate Uptime nc ac
gif0 10.255.255.1/32 P2P 00:00:04 unknown 02:10:24 0 0
vether0 172.16.1.1/24 BCKUP 00:00:09 active 02:10:24 1 1
Please, let me know if I'm doing something wrong/stupid or this is bug
somewhere in the stack.
I applied your configs to newly created VMs with 6.2-beta from today
(pf disabled). I see the same with tcpdump as you do.

Then I tried this:
- the OSPF adjacency comes up when I disable IPsec
- when I replace gif with gre ospfd is happy (with IPsec active)
(sysctl net.inet.gre.allow=1; mv /etc/hostname.{gif0,gre0})

I have a similar setup in production on 6.2 (with tunneldomain added to the
mix). This works.

To me this looks like a regression when gif is used with IPsec.
Post by Atanas Vladimirov
As a side note - eigrpd behaves the same way.
# dmesg
OpenBSD 6.3-beta (GENERIC.MP) #26: Fri Mar 2 22:56:04 MST 2018
real mem = 17132367872 (16338MB)
avail mem = 16606117888 (15836MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0: vendor American Megatrends Inc. version "2.2" date 02/20/2015
bios0: Thomas-Krenn.AG X9SCL/X9SCM
acpi0 at bios0: rev 2
acpi0: sleep states S0 S1 S4 S5
acpi0: tables DSDT FACP APIC FPDT MCFG HPET SSDT SPMI SSDT SSDT DMAR
acpi0: wakeup devices PS2K(S4) UAR1(S4) UAR2(S4) P0P1(S4) USB1(S4) USB2(S4)
USB3(S4) USB4(S4) USB5(S4) USB6(S4) USB7(S4) PXSX(S4) RP01(S4) PXSX(S4)
RP02(S4) PXSX(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT,MELTDOWN
cpu0: 256KB 64b/line 8-way L2 cache
acpitimer0: recalibrated TSC frequency 3300030874 Hz
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 100MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT,MELTDOWN
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT,MELTDOWN
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 6 (application processor)
cpu3: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT,MELTDOWN
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 0, core 3, package 0
cpu4 at mainbus0: apid 1 (application processor)
cpu4: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT,MELTDOWN
cpu4: 256KB 64b/line 8-way L2 cache
cpu4: smt 1, core 0, package 0
cpu5 at mainbus0: apid 3 (application processor)
cpu5: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT,MELTDOWN
cpu5: 256KB 64b/line 8-way L2 cache
cpu5: smt 1, core 1, package 0
cpu6 at mainbus0: apid 5 (application processor)
cpu6: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT,MELTDOWN
cpu6: 256KB 64b/line 8-way L2 cache
cpu6: smt 1, core 2, package 0
cpu7 at mainbus0: apid 7 (application processor)
cpu7: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT,MELTDOWN
cpu7: 256KB 64b/line 8-way L2 cache
cpu7: smt 1, core 3, package 0
ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 20, 24 pins
acpimcfg0 at acpi0 addr 0xe0000000, bus 0-255
acpihpet0 at acpi0: 14318179 Hz
acpihpet0: recalibrated TSC frequency 3300008959 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 7 (P0P1)
acpiprt2 at acpi0: bus 5 (RP01)
acpiprt3 at acpi0: bus -1 (RP02)
acpiprt4 at acpi0: bus -1 (RP03)
acpiprt5 at acpi0: bus -1 (RP04)
acpiprt6 at acpi0: bus 6 (RP05)
acpiprt7 at acpi0: bus -1 (RP06)
acpiprt8 at acpi0: bus -1 (RP07)
acpiprt9 at acpi0: bus -1 (RP08)
acpiprt10 at acpi0: bus 1 (PEG0)
acpiprt11 at acpi0: bus -1 (PEG1)
acpiprt12 at acpi0: bus -1 (PEG2)
acpiprt13 at acpi0: bus -1 (PEG3)
acpiec0 at acpi0: not present
acpipwrres0 at acpi0: FN00, resource for FAN0
acpipwrres1 at acpi0: FN01, resource for FAN1
acpipwrres2 at acpi0: FN02, resource for FAN2
acpipwrres3 at acpi0: FN03, resource for FAN3
acpipwrres4 at acpi0: FN04, resource for FAN4
acpitz0 at acpi0: critical temperature is 106 degC
acpitz1 at acpi0: critical temperature is 106 degC
"INT3F0D" at acpi0 not configured
"IPI0001" at acpi0 not configured
acpibtn0 at acpi0: PWRB
"PNP0C0B" at acpi0 not configured
"PNP0C0B" at acpi0 not configured
"PNP0C0B" at acpi0 not configured
"PNP0C0B" at acpi0 not configured
"PNP0C0B" at acpi0 not configured
acpivideo0 at acpi0: GFX0
acpivout0 at acpivideo0: DD01
acpivout at acpivideo0 not configured
acpivout at acpivideo0 not configured
acpivout at acpivideo0 not configured
acpivout at acpivideo0 not configured
acpivout at acpivideo0 not configured
acpivout at acpivideo0 not configured
acpivout at acpivideo0 not configured
ipmi at mainbus0 not configured
cpu0: Enhanced SpeedStep 3300 MHz: speeds: 3301, 3300, 3200, 3100, 2900,
2800, 2700, 2600, 2400, 2300, 2200, 2100, 2000, 1800, 1700, 1600 MHz
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel Xeon E3-1200 v2 Host" rev 0x09
ppb0 at pci0 dev 1 function 0 "Intel Core 3G PCIE" rev 0x09: msi
pci1 at ppb0 bus 1
ppb1 at pci1 dev 0 function 0 "IDT 89HPES12N3A" rev 0x0e
pci2 at ppb1 bus 2
ppb2 at pci2 dev 2 function 0 "IDT 89HPES12N3A" rev 0x0e
pci3 at ppb2 bus 3
em0 at pci3 dev 0 function 0 "Intel 82571EB" rev 0x06: apic 2 int 19,
address 00:15:17:bc:a9:65
em1 at pci3 dev 0 function 1 "Intel 82571EB" rev 0x06: apic 2 int 18,
address 00:15:17:bc:a9:64
ppb3 at pci2 dev 4 function 0 "IDT 89HPES12N3A" rev 0x0e
pci4 at ppb3 bus 4
em2 at pci4 dev 0 function 0 "Intel 82571EB" rev 0x06: apic 2 int 17,
address 00:15:17:bc:a9:67
em3 at pci4 dev 0 function 1 "Intel 82571EB" rev 0x06: apic 2 int 16,
address 00:15:17:bc:a9:66
em4 at pci0 dev 25 function 0 "Intel 82579LM" rev 0x05: msi, address
00:25:90:a1:c0:19
ehci0 at pci0 dev 26 function 0 "Intel 6 Series USB" rev 0x05: apic 2 int 16
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 configuration 1 interface 0 "Intel EHCI root hub" rev
2.00/1.00 addr 1
ppb4 at pci0 dev 28 function 0 "Intel 6 Series PCIE" rev 0xb5: msi
pci5 at ppb4 bus 5
ppb5 at pci0 dev 28 function 4 "Intel 6 Series PCIE" rev 0xb5: msi
pci6 at ppb5 bus 6
em5 at pci6 dev 0 function 0 "Intel 82574L" rev 0x00: msi, address
00:25:90:a1:c0:18
ehci1 at pci0 dev 29 function 0 "Intel 6 Series USB" rev 0x05: apic 2 int 23
usb1 at ehci1: USB revision 2.0
uhub1 at usb1 configuration 1 interface 0 "Intel EHCI root hub" rev
2.00/1.00 addr 1
ppb6 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0xa5
pci7 at ppb6 bus 7
vga1 at pci7 dev 3 function 0 "Matrox MGA G200eW" rev 0x0a
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
pcib0 at pci0 dev 31 function 0 "Intel C204 LPC" rev 0x05
ahci0 at pci0 dev 31 function 2 "Intel 6 Series AHCI" rev 0x05: msi, AHCI
1.3
ahci0: port 0: 3.0Gb/s
ahci0: port 2: 3.0Gb/s
scsibus1 at ahci0: 32 targets
sd0 at scsibus1 targ 0 lun 0: <ATA, WDC WD6401AALS-0, 05.0> SCSI3 0/direct
fixed naa.50014ee002288208
sd0: 610480MB, 512 bytes/sector, 1250263728 sectors
sd1 at scsibus1 targ 2 lun 0: <ATA, WDC WD6400AAKS-7, 01.0> SCSI3 0/direct
fixed naa.50014ee2adc0c1df
sd1: 610480MB, 512 bytes/sector, 1250263728 sectors
ichiic0 at pci0 dev 31 function 3 "Intel 6 Series SMBus" rev 0x05: apic 2
int 18
iic0 at ichiic0
sdtemp0 at iic0 addr 0x19: mcp98243
sdtemp1 at iic0 addr 0x1b: mcp98243
spdmem0 at iic0 addr 0x51: 8GB DDR3 SDRAM ECC PC3-12800 with thermal sensor
spdmem1 at iic0 addr 0x53: 8GB DDR3 SDRAM ECC PC3-12800 with thermal sensor
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
com2 at isa0 port 0x3e8/8 irq 5: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pckbd0 at pckbc0 (kbd slot)
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
wbsio0 at isa0 port 0x2e/2: NCT6776F rev 0x33
lm1 at wbsio0 port 0xa30/8: NCT6776F
vmm0 at mainbus0: VMX/EPT
uhub2 at uhub0 port 1 configuration 1 interface 0 "Intel Rate Matching Hub"
rev 2.00/0.00 addr 2
uhidev0 at uhub2 port 2 configuration 1 interface 0 "Winbond Electronics
Corp Hermon USB hidmouse Device" rev 1.10/0.01 addr 3
uhidev0: iclass 3/1
ums0 at uhidev0: 3 buttons, Z dir
wsmouse0 at ums0 mux 0
uhidev1 at uhub2 port 2 configuration 1 interface 1 "Winbond Electronics
Corp Hermon USB hidmouse Device" rev 1.10/0.01 addr 3
uhidev1: iclass 3/1
ukbd0 at uhidev1: 8 variable keys, 6 key codes
wskbd1 at ukbd0 mux 1
wskbd1: connecting to wsdisplay0
uhub3 at uhub1 port 1 configuration 1 interface 0 "Intel Rate Matching Hub"
rev 2.00/0.00 addr 2
uftdi0 at uhub3 port 3 configuration 1 interface 0 "FTDI FT232R USB UART"
rev 2.00/6.00 addr 3
ucom0 at uftdi0 portno 1
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
sd2 at scsibus3 targ 1 lun 0: <OPENBSD, SR RAID 1, 006> SCSI2 0/direct fixed
sd2: 610477MB, 512 bytes/sector, 1250258033 sectors
root on sd2a (391770920b7e6ed3.a) swap on sd2b dump on sd2b
Remi Locherer
2018-03-09 22:01:11 UTC
Permalink
Raw Message
Post by Remi Locherer
Post by Atanas Vladimirov
Hi,
I can't make OSPF to work on gif over IPsec.
# R1
[ns]~$ tcpdump -nei gif0
tcpdump: listening on gif0, link-type LOOP
23:19:29.181685 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid
192.168.1.1 area 0.0.0.1 [tos 0xc0] [ttl 1]
23:19:39.192025 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid
192.168.1.1 area 0.0.0.1 [tos 0xc0] [ttl 1]
23:19:49.202372 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid
192.168.1.1 area 0.0.0.1 [tos 0xc0] [ttl 1]
23:19:59.212730 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid
192.168.1.1 area 0.0.0.1 [tos 0xc0] [ttl 1]
23:20:09.223064 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid
192.168.1.1 area 0.0.0.1 [tos 0xc0] [ttl 1]
23:20:19.233393 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid
192.168.1.1 area 0.0.0.1 [tos 0xc0] [ttl 1]
# R2
[hodor]~$ tcpdump -nei gif0
tcpdump: listening on gif0, link-type LOOP
12:51:59.316704 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid 172.16.1.1
backbone [tos 0xc0] [ttl 1]
12:52:09.327002 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid 172.16.1.1
backbone [tos 0xc0] [ttl 1]
12:52:19.337314 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid 172.16.1.1
backbone [tos 0xc0] [ttl 1]
While on enc0 both hello's appears (not sure if `bad ip cksum` is the reason
# R1
[ns]~$ tcpdump -nvi enc0
tcpdump: listening on enc0, link-type ENC
12:24:37.625873 (authentic,confidential): SPI 0x11af4dae: 93.123.39.67 >
95.87.227.232: 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid 172.16.1.1
backbone E mask 255.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1]
(id 25841, len 64) (ttl 60, id 37752, len 84)
12:24:41.882173 (authentic,confidential): SPI 0x1a3fbc6d: 95.87.227.232 >
93.123.39.67: 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid 192.168.1.1
backbone E mask 255.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1]
(id 27818, len 64) (ttl 64, id 60563, len 84, bad ip cksum 32d7! -> c614)
12:24:47.636188 (authentic,confidential): SPI 0x11af4dae: 93.123.39.67 >
95.87.227.232: 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid 172.16.1.1
backbone E mask 255.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1]
(id 36067, len 64) (ttl 60, id 65348, len 84)
12:24:51.892467 (authentic,confidential): SPI 0x1a3fbc6d: 95.87.227.232 >
93.123.39.67: 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid 192.168.1.1
backbone E mask 255.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1]
(id 5127, len 64) (ttl 64, id 12476, len 84, bad ip cksum 201! -> 81ec)
12:24:57.646535 (authentic,confidential): SPI 0x11af4dae: 93.123.39.67 >
95.87.227.232: 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid 172.16.1.1
backbone E mask 255.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1]
(id 39220, len 64) (ttl 60, id 1938, len 84)
# R2
[hodor]~$ tcpdump -nvi enc0 | grep OSPF
tcpdump: listening on enc0, link-type ENC
12:28:57.894007 (authentic,confidential): SPI 0x11af4dae: 93.123.39.67 >
95.87.227.232: 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid 172.16.1.1
backbone E mask 255.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1]
(id 3667, len 64) (ttl 64, id 14037, len 84, bad ip cksum 2b6d! -> 7bd3)
12:29:02.151763 (authentic,confidential): SPI 0x1a3fbc6d: 95.87.227.232 >
93.123.39.67: 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid 192.168.1.1
backbone E mask 25
5.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1] (id 16974, len
64) (ttl 60, id 21648, len 84)
12:29:07.904315 (authentic,confidential): SPI 0x11af4dae: 93.123.39.67 >
95.87.227.232: 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid 172.16.1.1
backbone E mask 255
.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1] (id 45590, len 64)
(ttl 64, id 35262, len 84, bad ip cksum 2743! -> 28ea)
12:29:12.162049 (authentic,confidential): SPI 0x1a3fbc6d: 95.87.227.232 >
93.123.39.67: 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid 192.168.1.1
backbone E mask 25
5.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1] (id 19966, len
64) (ttl 60, id 3134, len 84)
12:29:17.914621 (authentic,confidential): SPI 0x11af4dae: 93.123.39.67 >
95.87.227.232: 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid 172.16.1.1
backbone E mask 255
.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1] (id 36161, len 64)
(ttl 64, id 53105, len 84, bad ip cksum fcb8! -> e336)
12:29:22.172468 (authentic,confidential): SPI 0x1a3fbc6d: 95.87.227.232 >
93.123.39.67: 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid 192.168.1.1
backbone E mask 25
5.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1] (id 36221, len
64) (ttl 60, id 29514, len 84)
If I set a static routes the regular traffic flows as it should.
# R1
[ns]~$ doas cat /etc/ipsec.conf
local_ip="95.87.227.232"
remote_ip="93.123.39.67"
ike esp transport from $local_ip to $remote_ip
# R2
[hodor]~$ doas cat /etc/ipsec.conf
local_ip="93.123.39.67"
remote_ip="95.87.227.232"
ike esp transport from $local_ip to $remote_ip
# R1
[ns]~$ doas cat /etc/hostname.gif0
up
mtu 1400
tunnel 95.87.227.232 93.123.39.67
inet 10.255.255.2/32
dest 10.255.255.1
# R2
[hodor]~$ doas cat /etc/hostname.gif0
up
mtu 1400
tunnel 93.123.39.67 95.87.227.232
inet 10.255.255.1/32
dest 10.255.255.2
# R1
[ns]~$ doas cat /etc/ospfd.conf
# $OpenBSD: ospfd.conf,v 1.1 2014/07/11 16:36:35 deraadt Exp $
# macros
password="secret"
# global configuration
router-id 192.168.1.1
fib-update yes
redistribute connected
# areas
area 0.0.0.0 {
interface gif0
}
# R2
[hodor]~$ doas cat /etc/ospfd.conf
# $OpenBSD: ospfd.conf,v 1.1 2014/07/11 16:36:35 deraadt Exp $
# macros
password="secret"
# global configuration
router-id 172.16.1.1
fib-update yes
redistribute connected
# areas
area 0.0.0.0 {
interface vether0
interface gif0
}
# R1
[ns]~$ ospfctl sh nei
ID Pri State DeadTime Address Iface Uptime
# R1
[ns]~$ ospfctl sh int
Interface Address State HelloTimer Linkstate Uptime nc ac
gif0 10.255.255.2/32 P2P 00:00:04 unknown 00:15:37 0 0
# R2
[hodor]~$ ospfctl sh nei
ID Pri State DeadTime Address Iface Uptime
172.16.1.9 1 FULL/DR 00:00:36 172.16.1.9 vether0 02:10:14
# R2
[hodor]~$ ospfctl sh int
Interface Address State HelloTimer Linkstate Uptime nc ac
gif0 10.255.255.1/32 P2P 00:00:04 unknown 02:10:24 0 0
vether0 172.16.1.1/24 BCKUP 00:00:09 active 02:10:24 1 1
Please, let me know if I'm doing something wrong/stupid or this is bug
somewhere in the stack.
I applied your configs to newly created VMs with 6.2-beta from today
(pf disabled). I see the same with tcpdump as you do.
- the OSPF adjacency comes up when I disable IPsec
- when I replace gif with gre ospfd is happy (with IPsec active)
(sysctl net.inet.gre.allow=1; mv /etc/hostname.{gif0,gre0})
I have a similar setup in production on 6.2 (with tunneldomain added to the
mix). This works.
To me this looks like a regression when gif is used with IPsec.
With below diff the setup works as expected: tcpdump shows OSPF hellos
on gif0 and ospfd sees the neighbour.

I don't think it's the correct fix though.


Index: if_gif.c
===================================================================
RCS file: /cvs/src/sys/net/if_gif.c,v
retrieving revision 1.112
diff -u -p -r1.112 if_gif.c
--- if_gif.c 28 Feb 2018 23:28:05 -0000 1.112
+++ if_gif.c 9 Mar 2018 20:52:46 -0000
@@ -745,8 +745,8 @@ gif_input(struct gif_tunnel *key, struct
}

/* XXX What if we run transport-mode IPsec to protect gif tunnel ? */
- if (m->m_flags & (M_AUTH | M_CONF))
- return (-1);
+ //if (m->m_flags & (M_AUTH | M_CONF))
+ // return (-1);

key->t_rtableid = m->m_pkthdr.ph_rtableid;
Atanas Vladimirov
2018-03-10 19:30:14 UTC
Permalink
Raw Message
Post by Remi Locherer
Post by Remi Locherer
Post by Atanas Vladimirov
Hi,
I can't make OSPF to work on gif over IPsec.
# R1
[ns]~$ tcpdump -nei gif0
tcpdump: listening on gif0, link-type LOOP
23:19:29.181685 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid
192.168.1.1 area 0.0.0.1 [tos 0xc0] [ttl 1]
23:19:39.192025 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid
192.168.1.1 area 0.0.0.1 [tos 0xc0] [ttl 1]
23:19:49.202372 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid
192.168.1.1 area 0.0.0.1 [tos 0xc0] [ttl 1]
23:19:59.212730 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid
192.168.1.1 area 0.0.0.1 [tos 0xc0] [ttl 1]
23:20:09.223064 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid
192.168.1.1 area 0.0.0.1 [tos 0xc0] [ttl 1]
23:20:19.233393 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid
192.168.1.1 area 0.0.0.1 [tos 0xc0] [ttl 1]
# R2
[hodor]~$ tcpdump -nei gif0
tcpdump: listening on gif0, link-type LOOP
12:51:59.316704 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid 172.16.1.1
backbone [tos 0xc0] [ttl 1]
12:52:09.327002 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid 172.16.1.1
backbone [tos 0xc0] [ttl 1]
12:52:19.337314 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid 172.16.1.1
backbone [tos 0xc0] [ttl 1]
While on enc0 both hello's appears (not sure if `bad ip cksum` is the reason
# R1
[ns]~$ tcpdump -nvi enc0
tcpdump: listening on enc0, link-type ENC
12:24:37.625873 (authentic,confidential): SPI 0x11af4dae: 93.123.39.67 >
95.87.227.232: 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid 172.16.1.1
backbone E mask 255.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1]
(id 25841, len 64) (ttl 60, id 37752, len 84)
12:24:41.882173 (authentic,confidential): SPI 0x1a3fbc6d: 95.87.227.232 >
93.123.39.67: 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid 192.168.1.1
backbone E mask 255.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1]
(id 27818, len 64) (ttl 64, id 60563, len 84, bad ip cksum 32d7! -> c614)
12:24:47.636188 (authentic,confidential): SPI 0x11af4dae: 93.123.39.67 >
95.87.227.232: 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid 172.16.1.1
backbone E mask 255.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1]
(id 36067, len 64) (ttl 60, id 65348, len 84)
12:24:51.892467 (authentic,confidential): SPI 0x1a3fbc6d: 95.87.227.232 >
93.123.39.67: 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid 192.168.1.1
backbone E mask 255.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1]
(id 5127, len 64) (ttl 64, id 12476, len 84, bad ip cksum 201! -> 81ec)
12:24:57.646535 (authentic,confidential): SPI 0x11af4dae: 93.123.39.67 >
95.87.227.232: 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid 172.16.1.1
backbone E mask 255.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1]
(id 39220, len 64) (ttl 60, id 1938, len 84)
# R2
[hodor]~$ tcpdump -nvi enc0 | grep OSPF
tcpdump: listening on enc0, link-type ENC
12:28:57.894007 (authentic,confidential): SPI 0x11af4dae: 93.123.39.67 >
95.87.227.232: 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid 172.16.1.1
backbone E mask 255.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1]
(id 3667, len 64) (ttl 64, id 14037, len 84, bad ip cksum 2b6d! -> 7bd3)
12:29:02.151763 (authentic,confidential): SPI 0x1a3fbc6d: 95.87.227.232 >
93.123.39.67: 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid 192.168.1.1
backbone E mask 25
5.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1] (id 16974, len
64) (ttl 60, id 21648, len 84)
12:29:07.904315 (authentic,confidential): SPI 0x11af4dae: 93.123.39.67 >
95.87.227.232: 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid 172.16.1.1
backbone E mask 255
.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1] (id 45590, len 64)
(ttl 64, id 35262, len 84, bad ip cksum 2743! -> 28ea)
12:29:12.162049 (authentic,confidential): SPI 0x1a3fbc6d: 95.87.227.232 >
93.123.39.67: 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid 192.168.1.1
backbone E mask 25
5.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1] (id 19966, len
64) (ttl 60, id 3134, len 84)
12:29:17.914621 (authentic,confidential): SPI 0x11af4dae: 93.123.39.67 >
95.87.227.232: 10.255.255.1 > 224.0.0.5: OSPFv2-hello 44: rtrid 172.16.1.1
backbone E mask 255
.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1] (id 36161, len 64)
(ttl 64, id 53105, len 84, bad ip cksum fcb8! -> e336)
12:29:22.172468 (authentic,confidential): SPI 0x1a3fbc6d: 95.87.227.232 >
93.123.39.67: 10.255.255.2 > 224.0.0.5: OSPFv2-hello 44: rtrid 192.168.1.1
backbone E mask 25
5.255.255.255 int 10 pri 1 dead 40 nbrs [tos 0xc0] [ttl 1] (id 36221, len
64) (ttl 60, id 29514, len 84)
If I set a static routes the regular traffic flows as it should.
# R1
[ns]~$ doas cat /etc/ipsec.conf
local_ip="95.87.227.232"
remote_ip="93.123.39.67"
ike esp transport from $local_ip to $remote_ip
# R2
[hodor]~$ doas cat /etc/ipsec.conf
local_ip="93.123.39.67"
remote_ip="95.87.227.232"
ike esp transport from $local_ip to $remote_ip
# R1
[ns]~$ doas cat /etc/hostname.gif0
up
mtu 1400
tunnel 95.87.227.232 93.123.39.67
inet 10.255.255.2/32
dest 10.255.255.1
# R2
[hodor]~$ doas cat /etc/hostname.gif0
up
mtu 1400
tunnel 93.123.39.67 95.87.227.232
inet 10.255.255.1/32
dest 10.255.255.2
# R1
[ns]~$ doas cat /etc/ospfd.conf
# $OpenBSD: ospfd.conf,v 1.1 2014/07/11 16:36:35 deraadt Exp $
# macros
password="secret"
# global configuration
router-id 192.168.1.1
fib-update yes
redistribute connected
# areas
area 0.0.0.0 {
interface gif0
}
# R2
[hodor]~$ doas cat /etc/ospfd.conf
# $OpenBSD: ospfd.conf,v 1.1 2014/07/11 16:36:35 deraadt Exp $
# macros
password="secret"
# global configuration
router-id 172.16.1.1
fib-update yes
redistribute connected
# areas
area 0.0.0.0 {
interface vether0
interface gif0
}
# R1
[ns]~$ ospfctl sh nei
ID Pri State DeadTime Address Iface Uptime
# R1
[ns]~$ ospfctl sh int
Interface Address State HelloTimer Linkstate Uptime nc ac
gif0 10.255.255.2/32 P2P 00:00:04 unknown 00:15:37 0 0
# R2
[hodor]~$ ospfctl sh nei
ID Pri State DeadTime Address Iface Uptime
172.16.1.9 1 FULL/DR 00:00:36 172.16.1.9 vether0 02:10:14
# R2
[hodor]~$ ospfctl sh int
Interface Address State HelloTimer Linkstate Uptime nc ac
gif0 10.255.255.1/32 P2P 00:00:04 unknown 02:10:24 0 0
vether0 172.16.1.1/24 BCKUP 00:00:09 active 02:10:24 1 1
Please, let me know if I'm doing something wrong/stupid or this is bug
somewhere in the stack.
I applied your configs to newly created VMs with 6.2-beta from today
(pf disabled). I see the same with tcpdump as you do.
- the OSPF adjacency comes up when I disable IPsec
- when I replace gif with gre ospfd is happy (with IPsec active)
(sysctl net.inet.gre.allow=1; mv /etc/hostname.{gif0,gre0})
I have a similar setup in production on 6.2 (with tunneldomain added to the
mix). This works.
To me this looks like a regression when gif is used with IPsec.
With below diff the setup works as expected: tcpdump shows OSPF hellos
on gif0 and ospfd sees the neighbour.
I don't think it's the correct fix though.
Index: if_gif.c
===================================================================
RCS file: /cvs/src/sys/net/if_gif.c,v
retrieving revision 1.112
diff -u -p -r1.112 if_gif.c
--- if_gif.c 28 Feb 2018 23:28:05 -0000 1.112
+++ if_gif.c 9 Mar 2018 20:52:46 -0000
@@ -745,8 +745,8 @@ gif_input(struct gif_tunnel *key, struct
}
/* XXX What if we run transport-mode IPsec to protect gif tunnel ? */
- if (m->m_flags & (M_AUTH | M_CONF))
- return (-1);
+ //if (m->m_flags & (M_AUTH | M_CONF))
+ // return (-1);
key->t_rtableid = m->m_pkthdr.ph_rtableid;
Hi Remi,

Thanks for confirming that there is an issue and I'm not doing something
wrong on my side.
I'll try the diff as soon as possible.
BR,
Atanas
David Gwynne
2018-03-13 06:28:40 UTC
Permalink
Raw Message
Post by Atanas Vladimirov
Post by Remi Locherer
With below diff the setup works as expected: tcpdump shows OSPF hellos
on gif0 and ospfd sees the neighbour.
I don't think it's the correct fix though.
Index: if_gif.c
===================================================================
RCS file: /cvs/src/sys/net/if_gif.c,v
retrieving revision 1.112
diff -u -p -r1.112 if_gif.c
--- if_gif.c 28 Feb 2018 23:28:05 -0000 1.112
+++ if_gif.c 9 Mar 2018 20:52:46 -0000
@@ -745,8 +745,8 @@ gif_input(struct gif_tunnel *key, struct
}
/* XXX What if we run transport-mode IPsec to protect gif tunnel ? */
- if (m->m_flags & (M_AUTH | M_CONF))
- return (-1);
+ //if (m->m_flags & (M_AUTH | M_CONF))
+ // return (-1);
key->t_rtableid = m->m_pkthdr.ph_rtableid;
Hi Remi,
Thanks for confirming that there is an issue and I'm not doing something wrong on my side.
I'll try the diff as soon as possible.
it isnt clear to me how ipsec and gif(4) are supposed to interact. on the one hand you have the gif(4) manpage saying this:

BUGS
There are many tunnelling protocol specifications, defined differently
from each other. gif may not interoperate with peers which are based on
different specifications, and are picky about outer header fields. For
example, you cannot usually use gif to talk with IPsec devices that use
IPsec tunnel mode.

so it's saying that ipsec tunnel mode and gif don't work, but then you have the code that remi is disabling saying that gif and ipsec transport dont work.

i can understand the issue since a decrypted esp packet looks a lot like the packets gif wants to handle. if we change to code or doco to make something work, which way should we go?

right now i would use gre inside ipsec transport mode, not gif. it has the benefit of working, and it is harder for traffic inside the tunnel to leak out of ipsec. more specifically, gif handles 3 ip protocols, ipv4, ipv6, and mpls, which are ip protocol numbers 4, 41, and 137 respectively. it is likely that people could set up ipsec to protect ipv4, but forget about ipv6 and mpls. if you then configure v6 or mpls on the gif interface, that traffic will leak.

gre on the other hand is a single ip protocol, so more straightforward to protect. there's also a very clear line in the sand between the inner and outer traffic, which esp tunnel and transport mode lack.

dlg
Remi Locherer
2018-03-13 09:24:43 UTC
Permalink
Raw Message
Post by David Gwynne
Post by Atanas Vladimirov
Post by Remi Locherer
With below diff the setup works as expected: tcpdump shows OSPF hellos
on gif0 and ospfd sees the neighbour.
I don't think it's the correct fix though.
Index: if_gif.c
===================================================================
RCS file: /cvs/src/sys/net/if_gif.c,v
retrieving revision 1.112
diff -u -p -r1.112 if_gif.c
--- if_gif.c 28 Feb 2018 23:28:05 -0000 1.112
+++ if_gif.c 9 Mar 2018 20:52:46 -0000
@@ -745,8 +745,8 @@ gif_input(struct gif_tunnel *key, struct
}
/* XXX What if we run transport-mode IPsec to protect gif tunnel ? */
- if (m->m_flags & (M_AUTH | M_CONF))
- return (-1);
+ //if (m->m_flags & (M_AUTH | M_CONF))
+ // return (-1);
key->t_rtableid = m->m_pkthdr.ph_rtableid;
Hi Remi,
Thanks for confirming that there is an issue and I'm not doing
something wrong on my side.
I'll try the diff as soon as possible.
it isnt clear to me how ipsec and gif(4) are supposed to interact. on
BUGS
There are many tunnelling protocol specifications, defined differently
from each other. gif may not interoperate with peers which are based on
different specifications, and are picky about outer header fields.
For
example, you cannot usually use gif to talk with IPsec devices that use
IPsec tunnel mode.
so it's saying that ipsec tunnel mode and gif don't work, but then you
have the code that remi is disabling saying that gif and ipsec
transport dont work.
BUGS is about one site using IPsec tunnel and the other site IPsec
transport with gif.

The problem reported by Atanas is about IPsec transport + gif on both
sites.
Post by David Gwynne
i can understand the issue since a decrypted esp packet looks a lot
like the packets gif wants to handle. if we change to code or doco to
make something work, which way should we go?
right now i would use gre inside ipsec transport mode, not gif. it has
the benefit of working,
yes ;-)
Post by David Gwynne
and it is harder for traffic inside the tunnel
to leak out of ipsec. more specifically, gif handles 3 ip protocols,
ipv4, ipv6, and mpls, which are ip protocol numbers 4, 41, and 137
respectively. it is likely that people could set up ipsec to protect
ipv4, but forget about ipv6 and mpls. if you then configure v6 or mpls
on the gif interface, that traffic will leak.
I don't see the big difference here between gif and gre. To prevent
traffic
leave your box unprotected you have to setup pf rules in both cases.
Post by David Gwynne
gre on the other hand is a single ip protocol, so more straightforward
to protect. there's also a very clear line in the sand between the
inner and outer traffic, which esp tunnel and transport mode lack.
But gre alone does not protect the traffic! You still need esp transport
for protection.

My main point about this is:
The setup described by Atanas used to work. There are setups out there
(including mine) that are operational for years and rely on OSPF working
via gif + esp transport. With the current state of gif these setups will
break.

Regards,
Remi
Marko Cupać
2018-03-13 11:39:39 UTC
Permalink
Raw Message
Hi,

sorry to hijack the thread, my question is not directly related, but
deals with same goal.

I have physical topology where datacentre has two carped firewalls,
while branch offices have single firewall each, with two uplinks:

isp2---em0
branchoffice1
datacenterA isp3---em1
em0
\ isp4---em0
carp0---isp1 INTERNET branchoffice2
/ isp5---em1
em0
datacenterB ispX---em0
branchofficeN
ispY---em1

I'd like to achieve two primary goals:
- each branch office has routes to both datacentre and all other branch
offices (OSPF?)
- each branch office uses em0 as primary link, fails over automatically
to em1 when em0 fails

I tried GRE tunnels from branch offices' both phsycal interfaces to
datacentres' carp interface, but this doesn't work (apparently gre is
not aware of carp and links go down when carp master changes). I din't
test two gre tunnels for each branch office's physical interface (one
to each carp member physical interface), as this seems too cumbersome to
maintain even if it worked.

Any advices?

Thank you in advance,
--
Before enlightenment - chop wood, draw water.
After enlightenment - chop wood, draw water.

Marko Cupać
https://www.mimar.rs/
Marc Peters
2018-03-13 10:56:12 UTC
Permalink
Raw Message
Post by David Gwynne
and it is harder for traffic inside the tunnel
to leak out of ipsec. more specifically, gif handles 3 ip protocols,
ipv4, ipv6, and mpls, which are ip protocol numbers 4, 41, and 137
respectively. it is likely that people could set up ipsec to protect
ipv4, but forget about ipv6 and mpls. if you then configure v6 or mpls
on the gif interface, that traffic will leak.
I don't see the big difference here between gif and gre. To prevent traffic
leave your box unprotected you have to setup pf rules in both cases.
Post by David Gwynne
gre on the other hand is a single ip protocol, so more straightforward
to protect. there's also a very clear line in the sand between the
inner and outer traffic, which esp tunnel and transport mode lack.
But gre alone does not protect the traffic! You still need esp transport
for protection.
The setup described by Atanas used to work. There are setups out there
(including mine) that are operational for years and rely on OSPF working
via gif + esp transport. With the current state of gif these setups will
break.
Our setup relies on gif interfaces and ipsec in transport mode, too. When setting this up, we had issues with the gre setup and gre packets itself brings in more headers, which would reduce the payload. That's why we are using ipsec (in transport mode) protected gif tunnels for ospf and bgp.

Best,
Marc
Maxim Bourmistrov
2018-03-13 16:33:11 UTC
Permalink
Raw Message
Post by Marc Peters
Post by David Gwynne
and it is harder for traffic inside the tunnel
to leak out of ipsec. more specifically, gif handles 3 ip protocols,
ipv4, ipv6, and mpls, which are ip protocol numbers 4, 41, and 137
respectively. it is likely that people could set up ipsec to protect
ipv4, but forget about ipv6 and mpls. if you then configure v6 or mpls
on the gif interface, that traffic will leak.
I don't see the big difference here between gif and gre. To prevent traffic
leave your box unprotected you have to setup pf rules in both cases.
Post by David Gwynne
gre on the other hand is a single ip protocol, so more straightforward
to protect. there's also a very clear line in the sand between the
inner and outer traffic, which esp tunnel and transport mode lack.
But gre alone does not protect the traffic! You still need esp transport
for protection.
The setup described by Atanas used to work. There are setups out there
(including mine) that are operational for years and rely on OSPF working
via gif + esp transport. With the current state of gif these setups will
break.
Our setup relies on gif interfaces and ipsec in transport mode, too. When setting this up, we had issues with the gre setup and gre packets itself brings in more headers, which would reduce the payload. That's why we are using ipsec (in transport mode) protected gif tunnels for ospf and bgp.
Best,
Marc
I moved over to etherip(4) some time ago. In transition to etherip, combo of etherip on one side and gif on another worked well.
I also remember announcement of gif(4) to be retired.

HISTORY
The gif device first appeared in WIDE hydrangea IPv6 kit.

Previously, gif supported RFC 3378 EtherIP tunnels over bridge(4)
interfaces. This is now handled by etherip(4).


//mxb
Marc Peters
2018-03-13 17:02:04 UTC
Permalink
Raw Message
Post by Maxim Bourmistrov
I moved over to etherip(4) some time ago. In transition to etherip, combo of etherip on one side and gif on another worked well.
I also remember announcement of gif(4) to be retired.
HISTORY
The gif device first appeared in WIDE hydrangea IPv6 kit.
Previously, gif supported RFC 3378 EtherIP tunnels over bridge(4)
interfaces. This is now handled by etherip(4).
Yeah, gif interfaces as bridge members, but we don't use bridging here.
David Gwynne
2018-03-13 22:40:23 UTC
Permalink
Raw Message
Post by Remi Locherer
With below diff the setup works as expected: tcpdump shows OSPF hellos
on gif0 and ospfd sees the neighbour.
I don't think it's the correct fix though.
functionally it is the correct fix.

when i reworked gif(4) in src/sys/net/if_gif.c r1.108, i merged the ipv4 and ipv6 input paths. the ipv6 input code had this check, but ipv4 did not. now it is applied to ipv4, but it is obviously wrong for both address families.

please commit the removal of this check, ok by me.

thank you to everyone for the but report and debugging. i'm sorry for taking so long to figure this out.

dlg
Post by Remi Locherer
Index: if_gif.c
===================================================================
RCS file: /cvs/src/sys/net/if_gif.c,v
retrieving revision 1.112
diff -u -p -r1.112 if_gif.c
--- if_gif.c 28 Feb 2018 23:28:05 -0000 1.112
+++ if_gif.c 9 Mar 2018 20:52:46 -0000
@@ -745,8 +745,8 @@ gif_input(struct gif_tunnel *key, struct
}
/* XXX What if we run transport-mode IPsec to protect gif tunnel ? */
- if (m->m_flags & (M_AUTH | M_CONF))
- return (-1);
+ //if (m->m_flags & (M_AUTH | M_CONF))
+ // return (-1);
key->t_rtableid = m->m_pkthdr.ph_rtableid;
csszep
2018-03-14 19:07:36 UTC
Permalink
Raw Message
Hi!

Will this fix be commit before 6.3 release?

Thx
csszep
Post by David Gwynne
Post by Remi Locherer
With below diff the setup works as expected: tcpdump shows OSPF hellos
on gif0 and ospfd sees the neighbour.
I don't think it's the correct fix though.
functionally it is the correct fix.
when i reworked gif(4) in src/sys/net/if_gif.c r1.108, i merged the ipv4
and ipv6 input paths. the ipv6 input code had this check, but ipv4 did not.
now it is applied to ipv4, but it is obviously wrong for both address
families.
please commit the removal of this check, ok by me.
thank you to everyone for the but report and debugging. i'm sorry for
taking so long to figure this out.
dlg
Post by Remi Locherer
Index: if_gif.c
===================================================================
RCS file: /cvs/src/sys/net/if_gif.c,v
retrieving revision 1.112
diff -u -p -r1.112 if_gif.c
--- if_gif.c 28 Feb 2018 23:28:05 -0000 1.112
+++ if_gif.c 9 Mar 2018 20:52:46 -0000
@@ -745,8 +745,8 @@ gif_input(struct gif_tunnel *key, struct
}
/* XXX What if we run transport-mode IPsec to protect gif tunnel ?
*/
Post by Remi Locherer
- if (m->m_flags & (M_AUTH | M_CONF))
- return (-1);
+ //if (m->m_flags & (M_AUTH | M_CONF))
+ // return (-1);
key->t_rtableid = m->m_pkthdr.ph_rtableid;
Claudio Jeker
2018-03-14 21:52:22 UTC
Permalink
Raw Message
Post by csszep
Hi!
Will this fix be commit before 6.3 release?
Yes something like this will need to be put into 6.3 since this is
currently a regression from 6.2.
Post by csszep
Post by David Gwynne
Post by Remi Locherer
With below diff the setup works as expected: tcpdump shows OSPF hellos
on gif0 and ospfd sees the neighbour.
I don't think it's the correct fix though.
functionally it is the correct fix.
when i reworked gif(4) in src/sys/net/if_gif.c r1.108, i merged the ipv4
and ipv6 input paths. the ipv6 input code had this check, but ipv4 did not.
now it is applied to ipv4, but it is obviously wrong for both address
families.
please commit the removal of this check, ok by me.
thank you to everyone for the but report and debugging. i'm sorry for
taking so long to figure this out.
dlg
Post by Remi Locherer
Index: if_gif.c
===================================================================
RCS file: /cvs/src/sys/net/if_gif.c,v
retrieving revision 1.112
diff -u -p -r1.112 if_gif.c
--- if_gif.c 28 Feb 2018 23:28:05 -0000 1.112
+++ if_gif.c 9 Mar 2018 20:52:46 -0000
@@ -745,8 +745,8 @@ gif_input(struct gif_tunnel *key, struct
}
/* XXX What if we run transport-mode IPsec to protect gif tunnel ?
*/
Post by Remi Locherer
- if (m->m_flags & (M_AUTH | M_CONF))
- return (-1);
+ //if (m->m_flags & (M_AUTH | M_CONF))
+ // return (-1);
key->t_rtableid = m->m_pkthdr.ph_rtableid;
--
:wq Claudio
Loading...