Very slow I/O under OpenBSD i386 on qemu-kvm from RHEL7rc

[Mikolaj Kucharski]

Hi,

Please CC me in any replies as I'm not subscribed to misc emails.

My main question is, do you experience similar slow I/O on OpenBSD i386
on your Qemu/KVM installations?

Are you aware of any problem with OpenBSD i386 under Qemu/KVM?

Not sure should this report go to RedHat, KVM or to OpenBSD, but I've
tested RHEL7rc with CentOS 6.5, NetBSD 6.1.4 installer and OpenBSD
current bsd.rd and only OpenBSD i386 seems to have slow I/O problem
under latest Enterprise Linux from RedHat. I've decided to report the
issue here. I've tested OpenBSD guest with IDE, SCSI and VirtIO disk.
SCSI disk currently don't seem to work at all with both amd64 and i386:


#### OpenBSD i386/scsi [test07] ####

# time dd if=/dev/zero of=/dev/sd0c bs=4096 count=10240
sd0(vioscsi0:0:0): Check Condition (error 0x70) on opcode 0x28
SENSE KEY: Illegal Request
ASC/ASCQ: ASC 0x24 ASCQ 0x00
dd: /dev/sd0c: Invalid argument
5+0 records in
4+0 records out
16384 bytes transferred in 0.093 secs (174834 bytes/sec)
0m0.25s real 0m0.00s user 0m0.15s system

# time dd if=/dev/zero of=/dev/rsd0c bs=4096 count=10240
sd0(vioscsi0:0:0): Check Condition (error 0x70) on opcode 0x2a
SENSE KEY: Illegal Request
ASC/ASCQ: ASC 0x24 ASCQ 0x00
dd: /dev/rsd0c: Invalid argument
42+0 records in
41+0 records out
167936 bytes transferred in 0.290 secs (578587 bytes/sec)
0m0.46s real 0m0.00s user 0m0.34s system


#### OpenBSD amd64/scsi [test08] ####

# time dd if=/dev/zero of=/dev/sd0c bs=4096 count=10240
sd0(vioscsi0:0:0): Check Condition (error 0x70) on opcode 0x28
SENSE KEY: Illegal Request
ASC/ASCQ: ASC 0x24 ASCQ 0x00
dd: /dev/sd0c: Invalid argument
5+0 records in
4+0 records out
16384 bytes transferred in 0.061 secs (265367 bytes/sec)
0m0.07s real 0m0.00s user 0m0.00s system

# time dd if=/dev/zero of=/dev/rsd0c bs=4096 count=10240
sd0(vioscsi0:0:0): Check Condition (error 0x70) on opcode 0x2a
SENSE KEY: Illegal Request
ASC/ASCQ: ASC 0x24 ASCQ 0x00
dd: /dev/rsd0c: Invalid argument
42+0 records in
41+0 records out
167936 bytes transferred in 0.081 secs (2063603 bytes/sec)
0m0.10s real 0m0.00s user 0m0.00s system


However I'm not chasing the SCSI problem, as VirtIO and IDE do work on
OpenBSD. VirtIO seems to be a bit faster, so I'm focusing all my tests
on VirtIO. If you look below at test02 you notice that to create 4GB
zero filled file takes 42 minutes. In comparision on Scientific Linux it
takes 5 minutes (also a bit slow, but not that much). On amd64 the same
operation is taking seconds.

+---------+---------+
| RHEL7 | SL6.5 |
+--------+---------+---------+
| i386 | 42min | 5min |
+--------+---------+---------+
| amd64 | 42sec | 28sec |
+--------+---------+---------+

I did tests on OpenBSD and NetBSD installers, however I have fully
installed i386 OpenBSD on RHEL7rc and it is also very slow. I'm
providing some information below, but if you need any more details,
please let me know.


==== DETAILS ====

rhel7rc> rpm -q kernel qemu-kvm
kernel-3.10.0-121.el7.x86_64
qemu-kvm-1.5.3-60.el7.x86_64

bsd.rd/amd64> dmesg | sed -e 2q
OpenBSD 5.5-current (RAMDISK_CD) #188: Fri Jun 13 13:01:15 MDT 2014
***@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/RAMDISK_CD

bsd.rd/i386> dmesg | sed -e 2q
OpenBSD 5.5-current (RAMDISK_CD) #169: Fri Jun 13 12:51:59 MDT 2014
***@i386.openbsd.org:/usr/src/sys/arch/i386/compile/RAMDISK_CD

# dmesg | grep sd0
sd0 at scsibus1 targ 0 lun 0: <VirtIO, Block Device, > SCSI3 0/direct fixed
sd0: 10240MB, 512 bytes/sector, 20971520 sectors

==== FILESYSTEM TESTS (4GB) ====

#### OpenBSD amd64/virtio (RHEL7rc) [test01] ####

# time dd if=/dev/zero of=/mnt/TEST bs=4096 count=1024x1024
1048576+0 records in
1048576+0 records out
4294967296 bytes transferred in 42.753 secs (100458532 bytes/sec)
0m42.79s real 0m0.07s user 0m5.77s system

#### OpenBSD i386/virtio (RHEL7rc) [test02] ####

# time dd if=/dev/zero of=/mnt/TEST bs=4096 count=1024x1024
1048576+0 records in
1048576+0 records out
4294967296 bytes transferred in 2592.993 secs (1656374 bytes/sec)
43m13.11s real 0m0.56s user 37m14.95s system


#### OpenBSD amd64/virtio (Scientific Linux 6.5) [test03] ####

# time dd if=/dev/zero of=/mnt/TEST bs=4096 count=1024x1024
1048576+0 records in
1048576+0 records out
4294967296 bytes transferred in 28.288 secs (151829547 bytes/sec)
0m28.29s real 0m0.22s user 0m6.78s system

#### OpenBSD i386/virtio (Scientific Linux 6.5) [test04] ####

# time dd if=/dev/zero of=/mnt/TEST bs=4096 count=1024x1024
1048576+0 records in
1048576+0 records out
4294967296 bytes transferred in 301.991 secs (14222149 bytes/sec)
5m2.00s real 0m0.61s user 4m15.20s system


#### NetBSD i386/virtio (RHEL7rc) [test05] ####

# dd if=/dev/zero of=/mnt/TEST bs=4096 count=1024x1024
1048576+0 records in
1048576+0 records out
4294967296 bytes transferred in 58.620 secs (73267951 bytes/sec)

#### NetBSD amd64/virtio (RHEL7rc) [test06] ####

# dd if=/dev/zero of=/mnt/TEST bs=4096 count=1024x1024
1048576+0 records in
1048576+0 records out
4294967296 bytes transferred in 58.278 secs (73697918 bytes/sec)


==== RAW DEVICE TESTS ====

#### OpenBSD i386/virtio (RHEL7rc) [test09] ####

# time dd if=/dev/zero of=/dev/sd0c bs=4096 count=1024x1024
1048576+0 records in
1048576+0 records out
4294967296 bytes transferred in 11793.765 secs (364173 bytes/sec)
196m42.80s real 0m0.83s user 104m36.56s system

# time dd if=/dev/zero of=/dev/rsd0c bs=4096 count=1024x1024
1048576+0 records in
1048576+0 records out
4294967296 bytes transferred in 9726.739 secs (441563 bytes/sec)
162m6.94s real 0m0.75s user 60m47.03s system


#### OpenBSD amd64/virtio (RHEL7rc) [test10] ####

# time dd if=/dev/zero of=/dev/sd0c bs=4096 count=1024x1024
1048576+0 records in
1048576+0 records out
4294967296 bytes transferred in 8600.437 secs (499389 bytes/sec)
143m20.50s real 0m1.55s user 0m58.67s system

# time dd if=/dev/zero of=/dev/rsd0c bs=4096 count=1024x1024
1048576+0 records in
1048576+0 records out
4294967296 bytes transferred in 819.963 secs (5237999 bytes/sec)
13m40.01s real 0m0.70s user 0m35.42s system


==== OTHER INFO ====

# dmesg
OpenBSD 5.5-current (RAMDISK_CD) #169: Fri Jun 13 12:51:59 MDT 2014
***@i386.openbsd.org:/usr/src/sys/arch/i386/compile/RAMDISK_CD
cpu0: QEMU Virtual CPU version 1.5.3 ("AuthenticAMD" 686-class, 512KB L2 cache) 2.61 GHz
cpu0: FPU,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,NXE,LONG,SSE3,CX16,LAHF,ABM,SSE4A,PERF
real mem = 536399872 (511MB)
avail mem = 519979008 (495MB)
mainbus0 at root
bios0 at mainbus0: AT/286+ BIOS, date 06/23/99, BIOS32 rev. 0 @ 0xfc672, SMBIOS rev. 2.4 @ 0xfdde0 (10 entries)
bios0: vendor Bochs version "Bochs" date 01/01/2011
bios0: Red Hat KVM
acpi0 at bios0: rev 0
acpi0: sleep states S5
acpi0: tables DSDT FACP SSDT APIC RSDT
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: apic clock running at 999MHz
ioapic0 at mainbus0: apid 0 pa 0xfec00000, version 11, 24 pins
acpiprt0 at acpi0: bus 0 (PCI0)
bios0: ROM list: 0xc0000/0x1000! 0xc1000/0xa00 0xc2000/0x2400 0xed000/0x3000!
pci0 at mainbus0 bus 0: configuration mode 1 (bios)
pchb0 at pci0 dev 0 function 0 "Intel 82441FX" rev 0x02
pcib0 at pci0 dev 1 function 0 "Intel 82371SB ISA" rev 0x00
pciide0 at pci0 dev 1 function 1 "Intel 82371SB IDE" rev 0x00: DMA, channel 0 wired to compatibility, channel 1 wired to compatibility
pciide0: channel 0 disabled (no drives)
atapiscsi0 at pciide0 channel 1 drive 0
scsibus0 at atapiscsi0: 2 targets
cd0 at scsibus0 targ 0 lun 0: <QEMU, QEMU DVD-ROM, 1.5.> ATAPI 5/cdrom removable
cd0(pciide0:1:0): using PIO mode 4, DMA mode 2
uhci0 at pci0 dev 1 function 2 "Intel 82371SB USB" rev 0x01: apic 0 int 11
"Intel 82371AB Power" rev 0x03 at pci0 dev 1 function 3 not configured
virtio0 at pci0 dev 3 function 0 "Qumranet Virtio Network" rev 0x00: Virtio Network Device
vio0 at virtio0: address 52:54:00:19:55:b8
virtio0: apic 0 int 11
virtio1 at pci0 dev 4 function 0 "Qumranet Virtio Storage" rev 0x00: Virtio Block Device
vioblk0 at virtio1
scsibus1 at vioblk0: 2 targets
sd0 at scsibus1 targ 0 lun 0: <VirtIO, Block Device, > SCSI3 0/direct fixed
sd0: 10240MB, 512 bytes/sector, 20971520 sectors
virtio1: apic 0 int 11
virtio2 at pci0 dev 5 function 0 "Qumranet Virtio Memory" rev 0x00: Virtio Memory Balloon Device
virtio2: no matching child driver; not configured
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com0: console
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard
pcdisplay0 at isa0 port 0x3d0/16 iomem 0xb8000/32768
wsdisplay0 at pcdisplay0 mux 1: console (80x25, vt100 emulation), using wskbd0
npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16
fdc0 at isa0 port 0x3f0/6 irq 6 drq 2
fd0 at fdc0 drive 1: density unknown
usb0 at uhci0: USB revision 1.0
uhub0 at usb0 "Intel UHCI root hub" rev 1.00/1.00 addr 1
softraid0 at root
scsibus2 at softraid0: 256 targets
root on rd0a swap on rd0b dump on rd0b


# sysctl
kern.osrelease=5.5
hw.machine=i386
hw.model=QEMU Virtual CPU version 1.5.3 ("AuthenticAMD" 686-class, 512KB L2 cache)
hw.product=KVM
hw.disknames=cd0:,sd0:819453647a0120e9,fd0:,rd0:70bbd6f4b5ade43a
hw.ncpufound=1

[Christiano F. Haesbaert]

Post by Mikolaj Kucharski
Hi,
Please CC me in any replies as I'm not subscribed to misc emails.
My main question is, do you experience similar slow I/O on OpenBSD i386
on your Qemu/KVM installations?
Are you aware of any problem with OpenBSD i386 under Qemu/KVM?
Not sure should this report go to RedHat, KVM or to OpenBSD, but I've
tested RHEL7rc with CentOS 6.5, NetBSD 6.1.4 installer and OpenBSD
current bsd.rd and only OpenBSD i386 seems to have slow I/O problem
under latest Enterprise Linux from RedHat. I've decided to report the
issue here. I've tested OpenBSD guest with IDE, SCSI and VirtIO disk.

The problem is the i386 ioapic/apic implementation and how OpenBSD
uses the lapic_tpr to block incoming interrupts.
Basically if you're using ioapic (and you are), OpenBSD maps the
lapic_tpr to the special TPR register to block interrupts everytime
you get an interrupt of raise it yourself by splraise()/spl* and so
on.

You don't suffer this on amd64 since the masking is done purely in
software. You can also verify this statement by disabling mpbios on
OpenBSD and falling back to the old pic controller, in this case you
don't use ioapic, and the pic code does the mask in software,
lapic_tpr is still the same variable being touched, but in this code
path, it's not mapped to the cpu TPR.

To have an ideia of the cost of touching the tpr:

real hardware: ~25 cycles.
kvm + flexpriority (cpu extension): ~2700 cycles.
kvm without flextpriority: 100k+ cycles.

So every interrupt you take needs to touch at least lapic_tpr twice,
which before would cost ~50cycles, and now it's more than ~200kcycles.

Of course these numbers are relevant to the machine I've tested, but
you get an idea on how much slower it is.

[Mikolaj Kucharski]

Post by Christiano F. Haesbaert

Post by Mikolaj Kucharski
My main question is, do you experience similar slow I/O on OpenBSD i386
on your Qemu/KVM installations?
Are you aware of any problem with OpenBSD i386 under Qemu/KVM?
Not sure should this report go to RedHat, KVM or to OpenBSD, but I've
tested RHEL7rc with CentOS 6.5, NetBSD 6.1.4 installer and OpenBSD
current bsd.rd and only OpenBSD i386 seems to have slow I/O problem
under latest Enterprise Linux from RedHat. I've decided to report the
issue here. I've tested OpenBSD guest with IDE, SCSI and VirtIO disk.

The problem is the i386 ioapic/apic implementation and how OpenBSD
uses the lapic_tpr to block incoming interrupts.

Giving what you wrote, where do you think the problem should be fixed?
On OpenBSD side, or on KVM side?

Post by Christiano F. Haesbaert
Basically if you're using ioapic (and you are), OpenBSD maps the
lapic_tpr to the special TPR register to block interrupts everytime
you get an interrupt of raise it yourself by splraise()/spl* and so
on.
You don't suffer this on amd64 since the masking is done purely in
software. You can also verify this statement by disabling mpbios on
OpenBSD and falling back to the old pic controller, in this case you

I cannot find how to enable 'the old pic controller' in libvirt with
qemu-kvm. Do you know by any chance how to enable it?

Post by Christiano F. Haesbaert
don't use ioapic, and the pic code does the mask in software,
lapic_tpr is still the same variable being touched, but in this code
path, it's not mapped to the cpu TPR.
real hardware: ~25 cycles.
kvm + flexpriority (cpu extension): ~2700 cycles.
kvm without flextpriority: 100k+ cycles.
So every interrupt you take needs to touch at least lapic_tpr twice,
which before would cost ~50cycles, and now it's more than ~200kcycles.
Of course these numbers are relevant to the machine I've tested, but
you get an idea on how much slower it is.

Thank you for the above explanation.

[Kevin Chadwick]

Post by Christiano F. Haesbaert
by disabling mpbios on

Post by Christiano F. Haesbaert
OpenBSD and falling back to the old pic controller, in this case you

I cannot find how to enable 'the old pic controller' in libvirt with
qemu-kvm. Do you know by any chance how to enable it?

I believe he means disabling mpbios at OpenBSD's boot or in boot.conf
means KVM will automatically fall back. Virtual hosting companies like
arpnetworks generally ask you to do this for OpenBSD.

boot -c
disable mpbios

checkout man boot.conf
--
_______________________________________________________________________

'Write programs that do one thing and do it well. Write programs to work
together. Write programs to handle text streams, because that is a
universal interface'

(Doug McIlroy)

In Other Words - Don't design like polkit or systemd
_______________________________________________________________________

[Mikolaj Kucharski]

Post by Kevin Chadwick

Post by Christiano F. Haesbaert
by disabling mpbios on

Post by Christiano F. Haesbaert
OpenBSD and falling back to the old pic controller, in this case you

I cannot find how to enable 'the old pic controller' in libvirt with
qemu-kvm. Do you know by any chance how to enable it?

I believe he means disabling mpbios at OpenBSD's boot or in boot.conf
means KVM will automatically fall back. Virtual hosting companies like
arpnetworks generally ask you to do this for OpenBSD.
boot -c
disable mpbios

Ah, I got confused. Yes, I'm aware of this, as I've seen this on the
list archives mentioned few times. I actually tested this, and I don't
see any difference. See at my below tests:


#### OpenBSD i386/virtio (default) [test12] ####

# time dd if=/dev/zero of=/tmp/TEST bs=4096 count=1024x1024
1048576+0 records in
1048576+0 records out
4294967296 bytes transferred in 3635.270 secs (1181471 bytes/sec)
60m35.50s real 0m1.24s user 54m15.15s system

#### OpenBSD i386/virtio (disable mpbios) [test13] ####

# time dd if=/dev/zero of=/tmp/TEST bs=4096 count=1024x1024
1048576+0 records in
1048576+0 records out
4294967296 bytes transferred in 3628.306 secs (1183739 bytes/sec)
60m28.49s real 0m1.33s user 54m8.30s system


On the archives I have seen recommendation to disable mpbios while
machine is slow in general, however I am experiencing only slow disk
I/O. I thought my problem is unrelated to mpbios.

With qemu-kvm from RHEL7 on bsd.sp there is no mpbios mentioned in
dmesg(8) (I didn't test bsd.mp). See dmesg output at the bottom of this
email. Also starting OpenBSD with mpbios disabled via boot_config(8)
ends up with:



--- dmesg.txt Sat Jun 14 15:49:02 2014
+++ disable-mpbios.txt Tue Jun 17 08:15:34 2014
@@ -4,6 +4,11 @@
cpu0: FPU,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,NXE,LONG,SSE3,CX16,LAHF,ABM,SSE4A,PERF
real mem = 536367104 (511MB)
avail mem = 515158016 (491MB)
+User Kernel Config
+UKC> disable mpbios
+368 mpbios0 disabled
+UKC> quit
+Continuing...
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
@@ -22,7 +27,7 @@
ioapic0 at mainbus0: apid 0 pa 0xfec00000, version 11, 24 pins
acpiprt0 at acpi0: bus 0 (PCI0)
acpicpu0 at acpi0
-bios0: ROM list: 0xc0000/0x1000! 0xc1000/0xa00 0xc2000/0x2400 0xed800/0x2800!
+bios0: ROM list: 0xc0000/0x1000! 0xc1000/0xa00 0xc2000/0x2400 0xed800/0x2800
pci0 at mainbus0 bus 0: configuration mode 1 (bios)
pchb0 at pci0 dev 0 function 0 "Intel 82441FX" rev 0x02
pcib0 at pci0 dev 1 function 0 "Intel 82371SB ISA" rev 0x00
@@ -32,11 +37,11 @@
uhci0 at pci0 dev 1 function 2 "Intel 82371SB USB" rev 0x01: apic 0 int 11
piixpm0 at pci0 dev 1 function 3 "Intel 82371AB Power" rev 0x03: apic 0 int 9
iic0 at piixpm0
-iic0: addr 0x1c 0f=00 words 00=9d87 01=9d87 02=9d87 03=9d87 04=9d87 05=9d87 06=9d87 07=9d87
-iic0: addr 0x1d 0f=00 words 00=9d87 01=9d87 02=9d87 03=9d87 04=9d87 05=9d87 06=9d87 07=9d87
-iic0: addr 0x4c 00=00 01=00 02=00 03=00 04=00 05=00 06=00 07=00 08=00 words 00=9d87 01=9d87 02=9d87 03=9d87 04=9d87 05=9d87 06=9d87 07=9d87
-iic0: addr 0x4d 3e=d1 48=d1 4a=d1 4e=d1 fc=d1 fe=d1 words 00=9d87 01=9d87 02=9d87 03=9d87 04=9d87 05=9d87 06=9d87 07=9d87
-iic0: addr 0x4e 00=00 01=00 02=00 03=00 04=00 05=00 06=00 07=00 08=00 3e=d1 48=d1 4a=d1 4e=d1 fc=d1 fe=d1 words 00=9d87 01=9d87 02=9d87 03=9d87 04=9d87 05=9d87 06=9d87 07=9d87
+iic0: addr 0x1c 0f=00 words 00=8fc5 01=8fc5 02=8fc5 03=8fc5 04=8fc5 05=8fc5 06=8fc5 07=8fc5
+iic0: addr 0x1d 0f=00 words 00=8fc5 01=8fc5 02=8fc5 03=8fc5 04=8fc5 05=8fc5 06=8fc5 07=8fc5
+iic0: addr 0x4c 00=00 01=00 02=00 03=00 04=00 05=00 06=00 07=00 08=00 words 00=8fc5 01=8fc5 02=8fc5 03=8fc5 04=8fc5 05=8fc5 06=8fc5 07=8fc5
+iic0: addr 0x4d 3e=d1 48=d1 4a=d1 4e=d1 fc=d1 fe=d1 words 00=8fc5 01=8fc5 02=8fc5 03=8fc5 04=8fc5 05=8fc5 06=8fc5 07=8fc5
+iic0: addr 0x4e 00=00 01=00 02=00 03=00 04=00 05=00 06=00 07=00 08=00 3e=d1 48=d1 4a=d1 4e=d1 fc=d1 fe=d1 words 00=8fc5 01=8fc5 02=8fc5 03=8fc5 04=8fc5 05=8fc5 06=8fc5 07=8fc5
virtio0 at pci0 dev 3 function 0 "Qumranet Virtio Network" rev 0x00: Virtio Network Device
vio0 at virtio0: address 52:54:00:12:34:70
virtio0: apic 0 int 11



Full, default dmesg:


OpenBSD 5.5-current (GENERIC) #162: Tue Jun 10 21:17:31 MDT 2014
***@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC
cpu0: QEMU Virtual CPU version 1.5.3 ("AuthenticAMD" 686-class, 512KB L2 cache) 2.61 GHz
cpu0: FPU,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,NXE,LONG,SSE3,CX16,LAHF,ABM,SSE4A,PERF
real mem = 536367104 (511MB)
avail mem = 515158016 (491MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: AT/286+ BIOS, date 06/23/99, BIOS32 rev. 0 @ 0xfc672, SMBIOS rev. 2.4 @ 0xfdde0 (10 entries)
bios0: vendor Bochs version "Bochs" date 01/01/2011
bios0: Red Hat KVM
acpi0 at bios0: rev 0
acpi0: sleep states S5
acpi0: tables DSDT FACP SSDT APIC RSDT
acpi0: wakeup devices
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 999MHz
ioapic0 at mainbus0: apid 0 pa 0xfec00000, version 11, 24 pins
acpiprt0 at acpi0: bus 0 (PCI0)
acpicpu0 at acpi0
bios0: ROM list: 0xc0000/0x1000! 0xc1000/0xa00 0xc2000/0x2400 0xed800/0x2800!
pci0 at mainbus0 bus 0: configuration mode 1 (bios)
pchb0 at pci0 dev 0 function 0 "Intel 82441FX" rev 0x02
pcib0 at pci0 dev 1 function 0 "Intel 82371SB ISA" rev 0x00
pciide0 at pci0 dev 1 function 1 "Intel 82371SB IDE" rev 0x00: DMA, channel 0 wired to compatibility, channel 1 wired to compatibility
pciide0: channel 0 disabled (no drives)
pciide0: channel 1 disabled (no drives)
uhci0 at pci0 dev 1 function 2 "Intel 82371SB USB" rev 0x01: apic 0 int 11
piixpm0 at pci0 dev 1 function 3 "Intel 82371AB Power" rev 0x03: apic 0 int 9
iic0 at piixpm0
iic0: addr 0x1c 0f=00 words 00=9d87 01=9d87 02=9d87 03=9d87 04=9d87 05=9d87 06=9d87 07=9d87
iic0: addr 0x1d 0f=00 words 00=9d87 01=9d87 02=9d87 03=9d87 04=9d87 05=9d87 06=9d87 07=9d87
iic0: addr 0x4c 00=00 01=00 02=00 03=00 04=00 05=00 06=00 07=00 08=00 words 00=9d87 01=9d87 02=9d87 03=9d87 04=9d87 05=9d87 06=9d87 07=9d87
iic0: addr 0x4d 3e=d1 48=d1 4a=d1 4e=d1 fc=d1 fe=d1 words 00=9d87 01=9d87 02=9d87 03=9d87 04=9d87 05=9d87 06=9d87 07=9d87
iic0: addr 0x4e 00=00 01=00 02=00 03=00 04=00 05=00 06=00 07=00 08=00 3e=d1 48=d1 4a=d1 4e=d1 fc=d1 fe=d1 words 00=9d87 01=9d87 02=9d87 03=9d87 04=9d87 05=9d87 06=9d87 07=9d87
virtio0 at pci0 dev 3 function 0 "Qumranet Virtio Network" rev 0x00: Virtio Network Device
vio0 at virtio0: address 52:54:00:12:34:70
virtio0: apic 0 int 11
virtio1 at pci0 dev 4 function 0 "Qumranet Virtio Storage" rev 0x00: Virtio Block Device
vioblk0 at virtio1
scsibus1 at vioblk0: 2 targets
sd0 at scsibus1 targ 0 lun 0: <VirtIO, Block Device, > SCSI3 0/direct fixed
sd0: 102400MB, 512 bytes/sector, 209715200 sectors
virtio1: apic 0 int 11
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com0: console
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard
pms0 at pckbc0 (aux slot)
pckbc0: using irq 12 for aux slot
wsmouse0 at pms0 mux 0
pcdisplay0 at isa0 port 0x3d0/16 iomem 0xb8000/32768
wsdisplay0 at pcdisplay0 mux 1: console (80x25, vt100 emulation), using wskbd0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16
fdc0 at isa0 port 0x3f0/6 irq 6 drq 2
fd0 at fdc0 drive 1: density unknown
usb0 at uhci0: USB revision 1.0
uhub0 at usb0 "Intel UHCI root hub" rev 1.00/1.00 addr 1
nvram: invalid checksum
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
root on sd0a (fcfbf65f39872058.a) swap on sd0b dump on sd0b
clock: unknown CMOS layout

[Gustav Fransson Nyvell]

Post by Mikolaj Kucharski

Post by Kevin Chadwick

Post by Christiano F. Haesbaert
by disabling mpbios on

Post by Christiano F. Haesbaert
OpenBSD and falling back to the old pic controller, in this case you

I cannot find how to enable 'the old pic controller' in libvirt with
qemu-kvm. Do you know by any chance how to enable it?

I believe he means disabling mpbios at OpenBSD's boot or in boot.conf
means KVM will automatically fall back. Virtual hosting companies like
arpnetworks generally ask you to do this for OpenBSD.
boot -c
disable mpbios

Ah, I got confused. Yes, I'm aware of this, as I've seen this on the
list archives mentioned few times. I actually tested this, and I don't
#### OpenBSD i386/virtio (default) [test12] ####
# time dd if=/dev/zero of=/tmp/TEST bs=4096 count=1024x1024
1048576+0 records in
1048576+0 records out
4294967296 bytes transferred in 3635.270 secs (1181471 bytes/sec)
60m35.50s real 0m1.24s user 54m15.15s system
#### OpenBSD i386/virtio (disable mpbios) [test13] ####
# time dd if=/dev/zero of=/tmp/TEST bs=4096 count=1024x1024
1048576+0 records in
1048576+0 records out
4294967296 bytes transferred in 3628.306 secs (1183739 bytes/sec)
60m28.49s real 0m1.33s user 54m8.30s system
On the archives I have seen recommendation to disable mpbios while
machine is slow in general, however I am experiencing only slow disk
I/O. I thought my problem is unrelated to mpbios.
With qemu-kvm from RHEL7 on bsd.sp there is no mpbios mentioned in
dmesg(8) (I didn't test bsd.mp). See dmesg output at the bottom of this
email. Also starting OpenBSD with mpbios disabled via boot_config(8)
--- dmesg.txt Sat Jun 14 15:49:02 2014
+++ disable-mpbios.txt Tue Jun 17 08:15:34 2014
@@ -4,6 +4,11 @@
cpu0: FPU,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,NXE,LONG,SSE3,CX16,LAHF,ABM,SSE4A,PERF
real mem = 536367104 (511MB)
avail mem = 515158016 (491MB)
+User Kernel Config
+UKC> disable mpbios
+368 mpbios0 disabled
+UKC> quit
+Continuing...
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
@@ -22,7 +27,7 @@
ioapic0 at mainbus0: apid 0 pa 0xfec00000, version 11, 24 pins
acpiprt0 at acpi0: bus 0 (PCI0)
acpicpu0 at acpi0
-bios0: ROM list: 0xc0000/0x1000! 0xc1000/0xa00 0xc2000/0x2400 0xed800/0x2800!
+bios0: ROM list: 0xc0000/0x1000! 0xc1000/0xa00 0xc2000/0x2400 0xed800/0x2800
pci0 at mainbus0 bus 0: configuration mode 1 (bios)
pchb0 at pci0 dev 0 function 0 "Intel 82441FX" rev 0x02
pcib0 at pci0 dev 1 function 0 "Intel 82371SB ISA" rev 0x00
@@ -32,11 +37,11 @@
uhci0 at pci0 dev 1 function 2 "Intel 82371SB USB" rev 0x01: apic 0 int 11
piixpm0 at pci0 dev 1 function 3 "Intel 82371AB Power" rev 0x03: apic 0 int 9
iic0 at piixpm0
-iic0: addr 0x1c 0f=00 words 00=9d87 01=9d87 02=9d87 03=9d87 04=9d87 05=9d87 06=9d87 07=9d87
-iic0: addr 0x1d 0f=00 words 00=9d87 01=9d87 02=9d87 03=9d87 04=9d87 05=9d87 06=9d87 07=9d87
-iic0: addr 0x4c 00=00 01=00 02=00 03=00 04=00 05=00 06=00 07=00 08=00 words 00=9d87 01=9d87 02=9d87 03=9d87 04=9d87 05=9d87 06=9d87 07=9d87
-iic0: addr 0x4d 3e=d1 48=d1 4a=d1 4e=d1 fc=d1 fe=d1 words 00=9d87 01=9d87 02=9d87 03=9d87 04=9d87 05=9d87 06=9d87 07=9d87
-iic0: addr 0x4e 00=00 01=00 02=00 03=00 04=00 05=00 06=00 07=00 08=00 3e=d1 48=d1 4a=d1 4e=d1 fc=d1 fe=d1 words 00=9d87 01=9d87 02=9d87 03=9d87 04=9d87 05=9d87 06=9d87 07=9d87
+iic0: addr 0x1c 0f=00 words 00=8fc5 01=8fc5 02=8fc5 03=8fc5 04=8fc5 05=8fc5 06=8fc5 07=8fc5
+iic0: addr 0x1d 0f=00 words 00=8fc5 01=8fc5 02=8fc5 03=8fc5 04=8fc5 05=8fc5 06=8fc5 07=8fc5
+iic0: addr 0x4c 00=00 01=00 02=00 03=00 04=00 05=00 06=00 07=00 08=00 words 00=8fc5 01=8fc5 02=8fc5 03=8fc5 04=8fc5 05=8fc5 06=8fc5 07=8fc5
+iic0: addr 0x4d 3e=d1 48=d1 4a=d1 4e=d1 fc=d1 fe=d1 words 00=8fc5 01=8fc5 02=8fc5 03=8fc5 04=8fc5 05=8fc5 06=8fc5 07=8fc5
+iic0: addr 0x4e 00=00 01=00 02=00 03=00 04=00 05=00 06=00 07=00 08=00 3e=d1 48=d1 4a=d1 4e=d1 fc=d1 fe=d1 words 00=8fc5 01=8fc5 02=8fc5 03=8fc5 04=8fc5 05=8fc5 06=8fc5 07=8fc5
virtio0 at pci0 dev 3 function 0 "Qumranet Virtio Network" rev 0x00: Virtio Network Device
vio0 at virtio0: address 52:54:00:12:34:70
virtio0: apic 0 int 11
OpenBSD 5.5-current (GENERIC) #162: Tue Jun 10 21:17:31 MDT 2014
cpu0: QEMU Virtual CPU version 1.5.3 ("AuthenticAMD" 686-class, 512KB L2 cache) 2.61 GHz
cpu0: FPU,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,NXE,LONG,SSE3,CX16,LAHF,ABM,SSE4A,PERF
real mem = 536367104 (511MB)
avail mem = 515158016 (491MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0: vendor Bochs version "Bochs" date 01/01/2011
bios0: Red Hat KVM
acpi0 at bios0: rev 0
acpi0: sleep states S5
acpi0: tables DSDT FACP SSDT APIC RSDT
acpi0: wakeup devices
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 999MHz
ioapic0 at mainbus0: apid 0 pa 0xfec00000, version 11, 24 pins
acpiprt0 at acpi0: bus 0 (PCI0)
acpicpu0 at acpi0
bios0: ROM list: 0xc0000/0x1000! 0xc1000/0xa00 0xc2000/0x2400 0xed800/0x2800!
pci0 at mainbus0 bus 0: configuration mode 1 (bios)
pchb0 at pci0 dev 0 function 0 "Intel 82441FX" rev 0x02
pcib0 at pci0 dev 1 function 0 "Intel 82371SB ISA" rev 0x00
pciide0 at pci0 dev 1 function 1 "Intel 82371SB IDE" rev 0x00: DMA, channel 0 wired to compatibility, channel 1 wired to compatibility
pciide0: channel 0 disabled (no drives)
pciide0: channel 1 disabled (no drives)
uhci0 at pci0 dev 1 function 2 "Intel 82371SB USB" rev 0x01: apic 0 int 11
piixpm0 at pci0 dev 1 function 3 "Intel 82371AB Power" rev 0x03: apic 0 int 9
iic0 at piixpm0
iic0: addr 0x1c 0f=00 words 00=9d87 01=9d87 02=9d87 03=9d87 04=9d87 05=9d87 06=9d87 07=9d87
iic0: addr 0x1d 0f=00 words 00=9d87 01=9d87 02=9d87 03=9d87 04=9d87 05=9d87 06=9d87 07=9d87
iic0: addr 0x4c 00=00 01=00 02=00 03=00 04=00 05=00 06=00 07=00 08=00 words 00=9d87 01=9d87 02=9d87 03=9d87 04=9d87 05=9d87 06=9d87 07=9d87
iic0: addr 0x4d 3e=d1 48=d1 4a=d1 4e=d1 fc=d1 fe=d1 words 00=9d87 01=9d87 02=9d87 03=9d87 04=9d87 05=9d87 06=9d87 07=9d87
iic0: addr 0x4e 00=00 01=00 02=00 03=00 04=00 05=00 06=00 07=00 08=00 3e=d1 48=d1 4a=d1 4e=d1 fc=d1 fe=d1 words 00=9d87 01=9d87 02=9d87 03=9d87 04=9d87 05=9d87 06=9d87 07=9d87
virtio0 at pci0 dev 3 function 0 "Qumranet Virtio Network" rev 0x00: Virtio Network Device
vio0 at virtio0: address 52:54:00:12:34:70
virtio0: apic 0 int 11
virtio1 at pci0 dev 4 function 0 "Qumranet Virtio Storage" rev 0x00: Virtio Block Device
vioblk0 at virtio1
scsibus1 at vioblk0: 2 targets
sd0 at scsibus1 targ 0 lun 0: <VirtIO, Block Device, > SCSI3 0/direct fixed
sd0: 102400MB, 512 bytes/sector, 209715200 sectors
virtio1: apic 0 int 11
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com0: console
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard
pms0 at pckbc0 (aux slot)
pckbc0: using irq 12 for aux slot
wsmouse0 at pms0 mux 0
pcdisplay0 at isa0 port 0x3d0/16 iomem 0xb8000/32768
wsdisplay0 at pcdisplay0 mux 1: console (80x25, vt100 emulation), using wskbd0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16
fdc0 at isa0 port 0x3f0/6 irq 6 drq 2
fd0 at fdc0 drive 1: density unknown
usb0 at uhci0: USB revision 1.0
uhub0 at usb0 "Intel UHCI root hub" rev 1.00/1.00 addr 1
nvram: invalid checksum
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
root on sd0a (fcfbf65f39872058.a) swap on sd0b dump on sd0b
clock: unknown CMOS layout

Hi,

OpenBSDs I/O speeds can be effected by various security schemes, also
OpenBSD is not the same OS as the others, it does not behave like maybe
you're used to, just get on with your work.

//Gustav

[Brad Smith]

Post by Mikolaj Kucharski

Post by Kevin Chadwick

Post by Christiano F. Haesbaert
by disabling mpbios on

Post by Christiano F. Haesbaert
OpenBSD and falling back to the old pic controller, in this case you

I cannot find how to enable 'the old pic controller' in libvirt with
qemu-kvm. Do you know by any chance how to enable it?

I believe he means disabling mpbios at OpenBSD's boot or in boot.conf
means KVM will automatically fall back. Virtual hosting companies like
arpnetworks generally ask you to do this for OpenBSD.
boot -c
disable mpbios

Ah, I got confused. Yes, I'm aware of this, as I've seen this on the
list archives mentioned few times. I actually tested this, and I don't

Because ACPI is in use which takes higher precedence over MP BIOS. You
have to disable acpimadt.

[Mike Larkin]

Post by Brad Smith

Post by Mikolaj Kucharski

Post by Kevin Chadwick

Post by Christiano F. Haesbaert
by disabling mpbios on

Post by Christiano F. Haesbaert
OpenBSD and falling back to the old pic controller, in this case you

I cannot find how to enable 'the old pic controller' in libvirt with
qemu-kvm. Do you know by any chance how to enable it?

I believe he means disabling mpbios at OpenBSD's boot or in boot.conf
means KVM will automatically fall back. Virtual hosting companies like
arpnetworks generally ask you to do this for OpenBSD.
boot -c
disable mpbios

Ah, I got confused. Yes, I'm aware of this, as I've seen this on the
list archives mentioned few times. I actually tested this, and I don't

Because ACPI is in use which takes higher precedence over MP BIOS. You
have to disable acpimadt.
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

Randomly disabling parts of the kernel is likely to cause other problems.

-ml

[Mikolaj Kucharski]

Mike,

Post by Mike Larkin

Post by Brad Smith
Because ACPI is in use which takes higher precedence over MP BIOS. You
have to disable acpimadt.

Randomly disabling parts of the kernel is likely to cause other problems.

I agree, but disabling mpbios and acpimadt makes a huge difference for me
on qemu-kvm-1.5.3-60.el7.x86_64:


#### OpenBSD i386/virtio (bsd.sp disable mpbios and acpimadt) ####

# time dd if=/dev/zero of=/tmp/TEST bs=4096 count=1024x1024
1048576+0 records in
1048576+0 records out
4294967296 bytes transferred in 18.524 secs (231854084 bytes/sec)
0m18.52s real 0m0.24s user 0m14.48s system


It takes now 18 seconds to run above dd(1), when previously it took 60
mintues. Thanks Christiano and Brad for the tips.


Do you guys think it's worth opening bug report with RedHat to get them
look into this, or is the problem more on OpenBSD side? Ideally I would
like to run unmodified OpenBSD kernel on my VMs.


$ diff -I'^iic0' dmesg.txt disable-mpbios-and-acpimadt.txt
--- dmesg.txt Sat Jun 14 15:49:02 2014
+++ disable-mpbios-and-acpimadt.txt Tue Jun 17 13:30:46 2014
@@ -4,6 +4,13 @@
cpu0: FPU,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,NXE,LONG,SSE3,CX16,LAHF,ABM,SSE4A,PERF
real mem = 536367104 (511MB)
avail mem = 515158016 (491MB)
+User Kernel Config
+UKC> disable mpbios
+368 mpbios0 disabled
+UKC> disable acpimadt
+501 acpimadt0 disabled
+UKC> quit
+Continuing...
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
@@ -15,22 +22,20 @@
acpi0: tables DSDT FACP SSDT APIC RSDT
acpi0: wakeup devices
acpitimer0 at acpi0: 3579545 Hz, 24 bits
-acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
-cpu0 at mainbus0: apid 0 (boot processor)
-mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
-cpu0: apic clock running at 999MHz
-ioapic0 at mainbus0: apid 0 pa 0xfec00000, version 11, 24 pins
acpiprt0 at acpi0: bus 0 (PCI0)
acpicpu0 at acpi0
+mpbios at bios0 function 0x0 not configured
bios0: ROM list: 0xc0000/0x1000! 0xc1000/0xa00 0xc2000/0x2400 0xed800/0x2800!
+cpu0 at mainbus0: (uniprocessor)
+mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
pci0 at mainbus0 bus 0: configuration mode 1 (bios)
pchb0 at pci0 dev 0 function 0 "Intel 82441FX" rev 0x02
pcib0 at pci0 dev 1 function 0 "Intel 82371SB ISA" rev 0x00
pciide0 at pci0 dev 1 function 1 "Intel 82371SB IDE" rev 0x00: DMA, channel 0 wired to compatibility, channel 1 wired to compatibility
pciide0: channel 0 disabled (no drives)
pciide0: channel 1 disabled (no drives)
-uhci0 at pci0 dev 1 function 2 "Intel 82371SB USB" rev 0x01: apic 0 int 11
-piixpm0 at pci0 dev 1 function 3 "Intel 82371AB Power" rev 0x03: apic 0 int 9
+uhci0 at pci0 dev 1 function 2 "Intel 82371SB USB" rev 0x01: irq 11
+piixpm0 at pci0 dev 1 function 3 "Intel 82371AB Power" rev 0x03: irq 9
iic0 at piixpm0
iic0: addr 0x1c 0f=00 words 00=d4d0 01=d4d0 02=d4d0 03=d4d0 04=d4d0 05=d4d0 06=d4d0 07=d4d0
iic0: addr 0x1d 0f=00 words 00=d4d0 01=d4d0 02=d4d0 03=d4d0 04=d4d0 05=d4d0 06=d4d0 07=d4d0
@@ -39,13 +44,13 @@
iic0: addr 0x4e 00=00 01=00 02=00 03=00 04=00 05=00 06=00 07=00 08=00 3e=d1 48=d1 4a=d1 4e=d1 fc=d1 fe=d1 words 00=9d87 01=9d87 02=9d87 03=9d87 04=9d87 05=9d87 06=9d87 07=9d87
virtio0 at pci0 dev 3 function 0 "Qumranet Virtio Network" rev 0x00: Virtio Network Device
vio0 at virtio0: address 52:54:00:12:34:70
-virtio0: apic 0 int 11
+virtio0: irq 11
virtio1 at pci0 dev 4 function 0 "Qumranet Virtio Storage" rev 0x00: Virtio Block Device
vioblk0 at virtio1
scsibus1 at vioblk0: 2 targets
sd0 at scsibus1 targ 0 lun 0: <VirtIO, Block Device, > SCSI3 0/direct fixed
sd0: 102400MB, 512 bytes/sector, 209715200 sectors
-virtio1: apic 0 int 11
+virtio1: irq 11
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo

[Mike Larkin]

Post by Mikolaj Kucharski
Mike,

Post by Mike Larkin

Post by Brad Smith
Because ACPI is in use which takes higher precedence over MP BIOS. You
have to disable acpimadt.

Randomly disabling parts of the kernel is likely to cause other problems.

I agree, but disabling mpbios and acpimadt makes a huge difference for me

Please don't bother filing any bug reports for a system where you've done this.

-ml

Post by Mikolaj Kucharski
#### OpenBSD i386/virtio (bsd.sp disable mpbios and acpimadt) ####
# time dd if=/dev/zero of=/tmp/TEST bs=4096 count=1024x1024
1048576+0 records in
1048576+0 records out
4294967296 bytes transferred in 18.524 secs (231854084 bytes/sec)
0m18.52s real 0m0.24s user 0m14.48s system
It takes now 18 seconds to run above dd(1), when previously it took 60
mintues. Thanks Christiano and Brad for the tips.
Do you guys think it's worth opening bug report with RedHat to get them
look into this, or is the problem more on OpenBSD side? Ideally I would
like to run unmodified OpenBSD kernel on my VMs.
$ diff -I'^iic0' dmesg.txt disable-mpbios-and-acpimadt.txt
--- dmesg.txt Sat Jun 14 15:49:02 2014
+++ disable-mpbios-and-acpimadt.txt Tue Jun 17 13:30:46 2014
@@ -4,6 +4,13 @@
cpu0: FPU,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,NXE,LONG,SSE3,CX16,LAHF,ABM,SSE4A,PERF
real mem = 536367104 (511MB)
avail mem = 515158016 (491MB)
+User Kernel Config
+UKC> disable mpbios
+368 mpbios0 disabled
+UKC> disable acpimadt
+501 acpimadt0 disabled
+UKC> quit
+Continuing...
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
@@ -15,22 +22,20 @@
acpi0: tables DSDT FACP SSDT APIC RSDT
acpi0: wakeup devices
acpitimer0 at acpi0: 3579545 Hz, 24 bits
-acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
-cpu0 at mainbus0: apid 0 (boot processor)
-mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
-cpu0: apic clock running at 999MHz
-ioapic0 at mainbus0: apid 0 pa 0xfec00000, version 11, 24 pins
acpiprt0 at acpi0: bus 0 (PCI0)
acpicpu0 at acpi0
+mpbios at bios0 function 0x0 not configured
bios0: ROM list: 0xc0000/0x1000! 0xc1000/0xa00 0xc2000/0x2400 0xed800/0x2800!
+cpu0 at mainbus0: (uniprocessor)
+mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
pci0 at mainbus0 bus 0: configuration mode 1 (bios)
pchb0 at pci0 dev 0 function 0 "Intel 82441FX" rev 0x02
pcib0 at pci0 dev 1 function 0 "Intel 82371SB ISA" rev 0x00
pciide0 at pci0 dev 1 function 1 "Intel 82371SB IDE" rev 0x00: DMA, channel 0 wired to compatibility, channel 1 wired to compatibility
pciide0: channel 0 disabled (no drives)
pciide0: channel 1 disabled (no drives)
-uhci0 at pci0 dev 1 function 2 "Intel 82371SB USB" rev 0x01: apic 0 int 11
-piixpm0 at pci0 dev 1 function 3 "Intel 82371AB Power" rev 0x03: apic 0 int 9
+uhci0 at pci0 dev 1 function 2 "Intel 82371SB USB" rev 0x01: irq 11
+piixpm0 at pci0 dev 1 function 3 "Intel 82371AB Power" rev 0x03: irq 9
iic0 at piixpm0
iic0: addr 0x1c 0f=00 words 00=d4d0 01=d4d0 02=d4d0 03=d4d0 04=d4d0 05=d4d0 06=d4d0 07=d4d0
iic0: addr 0x1d 0f=00 words 00=d4d0 01=d4d0 02=d4d0 03=d4d0 04=d4d0 05=d4d0 06=d4d0 07=d4d0
@@ -39,13 +44,13 @@
iic0: addr 0x4e 00=00 01=00 02=00 03=00 04=00 05=00 06=00 07=00 08=00 3e=d1 48=d1 4a=d1 4e=d1 fc=d1 fe=d1 words 00=9d87 01=9d87 02=9d87 03=9d87 04=9d87 05=9d87 06=9d87 07=9d87
virtio0 at pci0 dev 3 function 0 "Qumranet Virtio Network" rev 0x00: Virtio Network Device
vio0 at virtio0: address 52:54:00:12:34:70
-virtio0: apic 0 int 11
+virtio0: irq 11
virtio1 at pci0 dev 4 function 0 "Qumranet Virtio Storage" rev 0x00: Virtio Block Device
vioblk0 at virtio1
scsibus1 at vioblk0: 2 targets
sd0 at scsibus1 targ 0 lun 0: <VirtIO, Block Device, > SCSI3 0/direct fixed
sd0: 102400MB, 512 bytes/sector, 209715200 sectors
-virtio1: apic 0 int 11
+virtio1: irq 11
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
--
best regards
q#

[Christiano F. Haesbaert]

Post by Mikolaj Kucharski
Mike,

Post by Mike Larkin

Post by Brad Smith
Because ACPI is in use which takes higher precedence over MP BIOS. You
have to disable acpimadt.

Randomly disabling parts of the kernel is likely to cause other problems.

I agree, but disabling mpbios and acpimadt makes a huge difference for me
#### OpenBSD i386/virtio (bsd.sp disable mpbios and acpimadt) ####
# time dd if=/dev/zero of=/tmp/TEST bs=4096 count=1024x1024
1048576+0 records in
1048576+0 records out
4294967296 bytes transferred in 18.524 secs (231854084 bytes/sec)
0m18.52s real 0m0.24s user 0m14.48s system
It takes now 18 seconds to run above dd(1), when previously it took 60
mintues. Thanks Christiano and Brad for the tips.
Do you guys think it's worth opening bug report with RedHat to get them
look into this, or is the problem more on OpenBSD side? Ideally I would
like to run unmodified OpenBSD kernel on my VMs.

One solution is to rewrite the OpenBSD i386 interrupt handling code on
the apic case to use software masking, like amd64 and the i386 pic
code.

Post by Mikolaj Kucharski
$ diff -I'^iic0' dmesg.txt disable-mpbios-and-acpimadt.txt
--- dmesg.txt Sat Jun 14 15:49:02 2014
+++ disable-mpbios-and-acpimadt.txt Tue Jun 17 13:30:46 2014
@@ -4,6 +4,13 @@
cpu0: FPU,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,NXE,LONG,SSE3,CX16,LAHF,ABM,SSE4A,PERF
real mem = 536367104 (511MB)
avail mem = 515158016 (491MB)
+User Kernel Config
+UKC> disable mpbios
+368 mpbios0 disabled
+UKC> disable acpimadt
+501 acpimadt0 disabled
+UKC> quit
+Continuing...
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
@@ -15,22 +22,20 @@
acpi0: tables DSDT FACP SSDT APIC RSDT
acpi0: wakeup devices
acpitimer0 at acpi0: 3579545 Hz, 24 bits
-acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
-cpu0 at mainbus0: apid 0 (boot processor)
-mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
-cpu0: apic clock running at 999MHz
-ioapic0 at mainbus0: apid 0 pa 0xfec00000, version 11, 24 pins
acpiprt0 at acpi0: bus 0 (PCI0)
acpicpu0 at acpi0
+mpbios at bios0 function 0x0 not configured
bios0: ROM list: 0xc0000/0x1000! 0xc1000/0xa00 0xc2000/0x2400 0xed800/0x2800!
+cpu0 at mainbus0: (uniprocessor)
+mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
pci0 at mainbus0 bus 0: configuration mode 1 (bios)
pchb0 at pci0 dev 0 function 0 "Intel 82441FX" rev 0x02
pcib0 at pci0 dev 1 function 0 "Intel 82371SB ISA" rev 0x00
pciide0 at pci0 dev 1 function 1 "Intel 82371SB IDE" rev 0x00: DMA, channel 0 wired to compatibility, channel 1 wired to compatibility
pciide0: channel 0 disabled (no drives)
pciide0: channel 1 disabled (no drives)
-uhci0 at pci0 dev 1 function 2 "Intel 82371SB USB" rev 0x01: apic 0 int 11
-piixpm0 at pci0 dev 1 function 3 "Intel 82371AB Power" rev 0x03: apic 0 int 9
+uhci0 at pci0 dev 1 function 2 "Intel 82371SB USB" rev 0x01: irq 11
+piixpm0 at pci0 dev 1 function 3 "Intel 82371AB Power" rev 0x03: irq 9
iic0 at piixpm0
iic0: addr 0x1c 0f=00 words 00=d4d0 01=d4d0 02=d4d0 03=d4d0 04=d4d0 05=d4d0 06=d4d0 07=d4d0
iic0: addr 0x1d 0f=00 words 00=d4d0 01=d4d0 02=d4d0 03=d4d0 04=d4d0 05=d4d0 06=d4d0 07=d4d0
@@ -39,13 +44,13 @@
iic0: addr 0x4e 00=00 01=00 02=00 03=00 04=00 05=00 06=00 07=00 08=00 3e=d1 48=d1 4a=d1 4e=d1 fc=d1 fe=d1 words 00=9d87 01=9d87 02=9d87 03=9d87 04=9d87 05=9d87 06=9d87 07=9d87
virtio0 at pci0 dev 3 function 0 "Qumranet Virtio Network" rev 0x00: Virtio Network Device
vio0 at virtio0: address 52:54:00:12:34:70
-virtio0: apic 0 int 11
+virtio0: irq 11
virtio1 at pci0 dev 4 function 0 "Qumranet Virtio Storage" rev 0x00: Virtio Block Device
vioblk0 at virtio1
scsibus1 at vioblk0: 2 targets
sd0 at scsibus1 targ 0 lun 0: <VirtIO, Block Device, > SCSI3 0/direct fixed
sd0: 102400MB, 512 bytes/sector, 209715200 sectors
-virtio1: apic 0 int 11
+virtio1: irq 11
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
--
best regards
q#

[Kapetanakis Giannis]

Post by Christiano F. Haesbaert

Post by Mikolaj Kucharski
Do you guys think it's worth opening bug report with RedHat to get them
look into this, or is the problem more on OpenBSD side? Ideally I would
like to run unmodified OpenBSD kernel on my VMs.

One solution is to rewrite the OpenBSD i386 interrupt handling code on
the apic case to use software masking, like amd64 and the i386 pic
code.

Can someone "translate" this as
prefer amd64 arch over i386 when running on top of KVM?

regards,

G

[Markus Wernig]

Post by Brad Smith

Post by Kevin Chadwick
boot -c
disable mpbios

Because ACPI is in use which takes higher precedence over MP BIOS. You
have to disable acpimadt.

THANKS GUYS!!!!!!

This just "resolved" a blocker that had for 2 years prevented me from
upgrading my OpenBSD kvm guests to anything beyond 5.1!

/m