pflow packets before state expires

[Matt Hamilton]

Hi All,
We use pflow with pf to export packets to a collector for billing/monitoring
purposes. The problem we have is that someone at the weekend had a very
long running scp connection over several days that transferred a TB
of data. The data was not logged via pflow until the state expired, so
then showed a massive spike when the state expired.

Anyone know any way around this? Is it possible to get pf/pflow to
export more regularly? Or set some timeout? I'm guessing not due
to the architecture, and unless I force pf states to timeout then I'm
stuck? But thought I'd ask in case anyone knew of a way.

Thanks
-Matt

[sven falempin]

The manual say the information is extracted from the state table.
So you should have seen the info.

First: are you sure the information wasnt in the udp pflow packets ? maybe
the collector was wrong.
Second: man says <<The packet size and thus the maximum number of flows is
controlled by the mtu.>>

+

Post by Matt Hamilton
Hi All,
We use pflow with pf to export packets to a collector for
billing/monitoring
purposes. The problem we have is that someone at the weekend had a very
long running scp connection over several days that transferred a TB
of data. The data was not logged via pflow until the state expired, so
then showed a massive spike when the state expired.
Anyone know any way around this? Is it possible to get pf/pflow to
export more regularly? Or set some timeout? I'm guessing not due
to the architecture, and unless I force pf states to timeout then I'm
stuck? But thought I'd ask in case anyone knew of a way.
Thanks
-Matt

[Matt Hamilton]

Post by sven falempin
The manual say the information is extracted from the state table.
So you should have seen the info.
First: are you sure the information wasnt in the udp pflow packets ? maybe
the collector was wrong.
Second: man says <<The packet size and thus the maximum number of flows is
controlled by the mtu.>>

The problem is that (I believe) that the pflow packet is not generated until
the state expires from pf. In the case of the scp transfer I saw that was not
for several days. Meaning I had no accounting/reporting of this data
transfer until it ended and the state expired. At which point the entire
data transferred during that state's life was counted as if it happened now.

-Matt

[Henning Brauer]

[nonsense deleted]

Post by Matt Hamilton
The problem is that (I believe) that the pflow packet is not generated until
the state expires from pf. In the case of the scp transfer I saw that was not
for several days. Meaning I had no accounting/reporting of this data
transfer until it ended and the state expired.

correct.

Post by Matt Hamilton
At which point the entire
data transferred during that state's life was counted as if it happened now.

This I'd call a visualization bug; but that doesn't change too much
here.