NetMap in OpenBSD

[Mikael]

NetMap (http://info.iet.unipi.it/~luigi/netmap/) in OpenBSD would be a
great idea. It's a simple API for solving an important problem, at least
its core parts.

Is OBSD's kernel structure suited as it is for NetMap?

What's the interest out there for NetMap on OBSD?

Thanks,
Mikael

[Henning Brauer]

Post by Mikael
NetMap (http://info.iet.unipi.it/~luigi/netmap/) in OpenBSD would be a
great idea.

for what?
to create even more broken userland networking stuff?

We kinda like our stack.

Post by Mikael
What's the interest out there for NetMap on OBSD?

roughly somewhere between 0 and zero.

[Mikael]

Dear Henning,

Thank you for your thoughtful response.

Post by Henning Brauer

Post by Mikael
NetMap (http://info.iet.unipi.it/~luigi/netmap/) in OpenBSD would be a
great idea.

We kinda like our stack.
Of course, OBSD has a very good stack as it is, but it has no NetMap
functionality i.e. there's no way for a userland application to do high
speed packet-level IO.

Post by Henning Brauer
for what?

There is a whole world of need of network monitoring and manipulation and
other specialized networking software.

The benefit with doing this on OBSD is that it's more robust than other
systems and at least in my world that's hard currency.



Chris got NetMap to compile on OBSD though I have no clue if/how well
present subsystems deliver for it.


Mikael

[sven falempin]

Post by Mikael
Dear Henning,
Thank you for your thoughtful response.

Post by Henning Brauer

Post by Mikael
NetMap (http://info.iet.unipi.it/~luigi/netmap/) in OpenBSD would be a
great idea.

We kinda like our stack.
Of course, OBSD has a very good stack as it is, but it has no NetMap
functionality i.e. there's no way for a userland application to do high
speed packet-level IO.

Post by Henning Brauer
for what?

There is a whole world of need of network monitoring and manipulation and
other specialized networking software.
The benefit with doing this on OBSD is that it's more robust than other
systems and at least in my world that's hard currency.
Chris got NetMap to compile on OBSD though I have no clue if/how well
present subsystems deliver for it.
Mikael

IMHO netmap is for hypervisor, OpenBSD is not an hypervisor.

[Henning Brauer]

Post by Mikael

Post by Henning Brauer

Post by Mikael
NetMap (http://info.iet.unipi.it/~luigi/netmap/) in OpenBSD would be a
great idea.

We kinda like our stack.

Of course, OBSD has a very good stack as it is, but it has no NetMap
functionality

yeah, and that is good. netmap bypasses teh stack and you look at
reimplementing the stack in userland, repeating mistakes, bugs and
whatnot from many decades.

Post by Mikael
i.e. there's no way for a userland application to do high speed
packet-level IO.

there are plenty of methods actually.

userland reimplementing the stack for the sake of speed is beyond
idiotic. i rather spend the time to make the stack even faster than it
already is.

Post by Mikael
There is a whole world of need of network monitoring and manipulation and
other specialized networking software.

I read a collection of buzzwords with nothing specific.

"A solution in dire need of a problem."

[Mikael]

Post by Henning Brauer

Post by Mikael
Of course, OBSD has a very good stack as it is, but it has no NetMap
functionality

yeah, and that is good. netmap bypasses teh stack and you look at
reimplementing the stack in userland, repeating mistakes, bugs and
whatnot from many decades.

Post by Mikael
i.e. there's no way for a userland application to do high speed
packet-level IO.

there are plenty of methods actually.

Like what?

Post by Henning Brauer
userland reimplementing the stack[...]

I didn't necessarily/specifically suggest that.

Post by Henning Brauer
There is a whole world of need of network monitoring and manipulation and

Post by Mikael
other specialized networking software.

I read a collection of buzzwords with nothing specific.
"A solution in dire need of a problem."

Will be more clear on this one following your response. Last for completing
reflections -



Most devices in a system can be accessed with good performance from
userland as it is now, for instance block devices, USB, serial ports, video
and audio.

Ethernet is a rare exception and NetMap solved this in a neat way -

Prior to NetMap, those who wanted to make high-performance ethernet IO in
userland would run their app as root and effectively implement NIC hardware
drivers in userland. NetMap generalized this entire problem to one
hardware-agnostic interface.

[Jan Stary]

Post by Mikael
Most devices in a system can be accessed with good performance from
userland as it is now, for instance block devices, USB, serial ports, video
and audio.

Repeat after me: userland is not supposed to "access devices".
It is supposed to talk to a defined interface and let thr kernel
talk to the device.

Post by Mikael
Ethernet is a rare exception and NetMap solved this in a neat way -
Prior to NetMap, those who wanted to make high-performance ethernet IO in
userland would run their app as root and effectively implement NIC hardware
drivers in userland. NetMap generalized this entire problem to one
hardware-agnostic interface.

Repeat after me: it is kernel's job to
"make high-performance ethernet IO".

[Henning Brauer]

Post by Mikael

Post by Henning Brauer

Post by Mikael
i.e. there's no way for a userland application to do high speed
packet-level IO.

there are plenty of methods actually.

Like what?

bpf, for example.

but since you still don't mention what problem you're trying to
solve...

Post by Mikael

Post by Henning Brauer
userland reimplementing the stack[...]

I didn't necessarily/specifically suggest that.

but that's what you effectively HAVE TO DO with netmap, unless you're
creating some layer2 bridge (which belongs in kernel space), or just
want to listen (there is bpf for that).

Post by Mikael

Post by Henning Brauer
There is a whole world of need of network monitoring and manipulation and

Post by Mikael
other specialized networking software.

I read a collection of buzzwords with nothing specific.
"A solution in dire need of a problem."

Will be more clear on this one following your response. Last for completing
reflections -
Most devices in a system can be accessed with good performance from
userland as it is now, for instance block devices, USB, serial ports, video
and audio.
Ethernet is a rare exception and NetMap solved this in a neat way -

bolloks.
foremost, in almost all cases you don't speak ethernet, you speak IP
(just like you don't speak USB to access a umass in userland).

Post by Mikael
Prior to NetMap, those who wanted to make high-performance ethernet IO in
userland would run their app as root and effectively implement NIC hardware
drivers in userland. NetMap generalized this entire problem to one
hardware-agnostic interface.

ok, still bla bla without a use case, not even speaking about a valid
one or one that is common enough to push yet another network subsystem
into the kernel.

still stinks like a solution in need of a problem.

netmap is luigi's research framework, and he used it for some cool
research an sure will do so more in the future. no more, no less.

All this stack bypassing and (partial and buggy) reimplementation in
userland baloony has to stop. Introducing interop and security issues
just to look a little better in made up microbenchmarks, without any
real world relevance, what an awesome deal.

The time needed to "port" netmap (which includes touching EVERY NIC
driver) plus the time for the fruitless attempt to get IP processing
close to right in userland to make a specific application a little
faster is spent much better improving the network stack itself - for
all applications.

[Henning Brauer]

Post by Henning Brauer
netmap is luigi's research framework, and he used it for some cool
research an sure will do so more in the future. no more, no less.

I should clarify: I am aware of a few use cases that profit enormously
from netmap.

Let's look at what netmap really is, pardon some slight inaccuracies
for the sake of clarity: netmap is a ring buffer shoveling raw
packets from the NIC's RX ring into userland and vice versa (to the TX
ring of course). As such it is similar to BPF, but bpf does more,
which is one reason why netmap is faster.

Now these use cases are relatively rare; introducing yet another
interface that is somewhat like an existing doesn't come for free -
neither is the porting work done by sending an email to misc, nor is
maintainance free. IPX and appletalk have their use cases too, and yet
we deleted them - because they are to rare to justify the maintainance
burden.

Now if you want to spend time on improving these few use cases, that
time is much better spent improving the existing interface imo - with
all the existing consumers profiting. There's plenty of room without
changing anything userland visible, esp. the no-filter case can
probably speed up significantly without too much effort. Might even
bring some ideas from netmap in (some would probably require minimal
adjustments for existing consumers to profit, still way less effort
than converting to a new interface).

And let me repeat: all attempts to reimplement the IP stack in
userland are not smart, heck, even dangerous. Not all cases fall into
that category, but working w/ and in the network stack for more than a
decade, I keep thinking I have a pretty good idea on what "great" ideas
some people end up with.

Luigi and I discussed netmap before, at length. We even mostly agree,
it's for some very specific cases only. We disagree on the question
whether it belongs into a general purpose OS kernel, plus, as I keep
mentioning - it's not done by porting it, there is ongoing
maintainance - our manpower is limited and we're not remotely out of
ideas on how to improve networking for everyone.

Now pardon me, beer is calling :)

[Raimundo Santos]

Sorry, replied to fast and to OP only.

Below is one use case and a lot o things that Henning have said, put from
my point of view.
---------- Forwarded message ----------
From: Raimundo Santos <***@gmail.com>
Date: 14 October 2014 15:02
Subject: Re: NetMap in OpenBSD

Post by Mikael

Post by Henning Brauer
userland reimplementing the stack[...]

I didn't necessarily/specifically suggest that.

The only case I can see to not reimplement full stack is working on pure
Ethernet. All other "really nice" one can do with TCP/IP are sadly going to
be reimplemented.

This is how netmap works, barely: put packets in ring buffers, bypassing
all the neat work of years in the OS network stack. How do you route a
packet within netmap logic? How do you check for source or destiny
addresses or TCP/UDP ports? You need to reimplement it on your own program,
and do that for EVERY program using netmap.

Post by Mikael

Post by Henning Brauer
There is a whole world of need of network monitoring and manipulation

and

Post by Mikael

Post by Henning Brauer

Post by Mikael
other specialized networking software.

I read a collection of buzzwords with nothing specific.
"A solution in dire need of a problem."

Here I see the limit of a general purpose OS. Well, lets deal with all the
corner cases, and all the possibilities, and lets create a general purpose
OS that is a specific purpose for everyone who uses it. Makes no sense to
me. Specific needs that are not covered by the general facilities of such
an OS must be covered by specific work of who needs it. You can even make a
profitable product of this work. :)

Bypass years of correct and conscious work to make all the stack more
secure just because the needs of a few are for speed? It is a bad choice.

netmap have one thing that really interests me: the ability to enforce
specific per-ip bandwidth with dummynet, but at the cost of doing this with
netmap-ipfw, reimplementing all the needed stack parts.

Why, my sacred believes, WHY?! So, instead of improving that stack to do a
free for all, correct and conscious speed up, lets do it by reimplementing
the "needed parts" in every application.

sosplice(9) may serve us with a starting point to that "really fast" things
of zero-copy hype.

http://www.openbsd.org/papers/eurobsdcon_2013_sosplice-slides.pdf

Summarizing: netmap bypasses ALL the OS network stack. Period. Therefore,
you must reimplement such things.

Best regards,
Raimundo Santos