Discussion:
Addblock + Badhost blocking via unbound(8) and pf anchors
(too old to reply)
Jordan Geoghegan
2017-12-29 22:50:31 UTC
Permalink
Hi everyone,

Due to the number of people who have requested my add-blocking scripts,
I figured I would also post them to @misc so anyone can easily enjoy
network-wide bad-host/add-blocking.

I won't go into detail on how to set up routing/dhcp/unbound/anchors
etc, for that see: https://www.openbsd.org/faq/pf/example1.html

I've included some example files from my an Edgerouter I have set up .
They are trimmed down for brevities sake; the conf files are not
production ready, these are merely examples.

This setup is easily customizable, if you come across any other block
lists you prefer, then they can be dropped in no problem. I chose to use
solely the StevenBlack hosts file because it is a master list compiled
from all the major banlists found in popular blocking products such as
uBlock Origin, Addblock Plus et al. I also chose this file because it is
filtered for duplicates as unbound(8) is said to struggle when there are
redundancies in the blocklists, I'm told -- though I've never had any issue.

You're going to have to read the scripts and create the directories the
scripts are calling and edit the anchor macros to fit your interface
layout (I doubt everyone here is running cnmac0 as egress) and also will
have to make the scripts executable and set them to run at regular
intervals with crontab, ideally nightly.

I didn't make these scripts intelligent because I figured it was simpler
to just run mkdir once rather than add extra lines to the script.

I know the pf.conf is fairly long, I thought I would show an example of
my prio and queing setup as an example, or conversely to see if anyone
can poke any holes in it.

All the relevant bits regarding the anchors and blocklists are found at
the end of the pf.conf file. See below that for the anchor conf files
we're calling as well.

Hope this helps,

Jordan Geoghegan


First, the scripts:

*DNS addblock script:*

StevenBlack.sh:

cd /var/unbound/etc/banlist && \
ftp https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts && \
cat hosts | grep '^0\.0\.0\.0' | awk '{print "local-zone: \""$2"\" redirect\nlocal-data: \""$2" A 0.0.0.0\""}' > ads.conf
rcctl reload unbound

###

*IP based malicious IP blocking:*

banlist.sh:

cd /etc/blocklist && ftp https://www.binarydefense.com/banlist.txt\
&& ftp https://rules.emergingthreats.net/blockrules/compromised-ips.txt\
&& ftp https://rules.emergingthreats.net/fwrules/emerging-Block-IPs.txt\
&& ftp https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/firehol_level3.netset\
&& pfctl -a banlist -f /etc/banlist.conf

###

As you can see, we are going to have to make an anchor in pf called
'banlist' and modify the unbound.conf to load our banlist 'ads.conf'

If that's all you need, then you're pretty much good to go. If you would
like to see my example conf files, see below.

*


Example unbound.conf:*

# $OpenBSD: unbound.conf,v 1.7 2016/03/30 01:41:25 sthen Exp $

server:
interface: 172.17.17.1
interface: 127.0.0.1
access-control: 172.17.17.0/24 allow
access-control: 172.17.0.0/24 allow
do-not-query-localhost: no
hide-identity: yes
hide-version: yes
include: /var/unbound/etc/banlist/ads.conf

forward-zone:
name: "."
forward-addr: UR.DNS.GO.HERE
forward-addr: UR.DNS.GO.HERE

###


*Example pf.conf:*

# $OpenBSD: pf.conf,v 1.54 2014/08/23 05:49:42 deraadt Exp $
#
# See pf.conf(5) and /etc/examples/pf.conf

# By default, do not permit remote connections to X11
block return in on ! lo0 proto tcp to port 6000:6010
#
ext_if="{ cnmac0 }"
int_if="{ cnmac1 cnmac2 }"
lan_if="{ cnmac1 }"
wifi_if="{ cnmac2 }"
goodguys="{ 172.17.17.0/24 }"
wifiguys="{ 172.17.0.0/24 }"
chromecast="{ 172.17.0.12 172.17.0.13 172.17.0.23 }"
xbox360="{ 172.17.0.19 }"
printer="{ 172.17.0.17 }"
Jordan="{ XXX.XX.XXX.XX }"

table <martians> { 0.0.0.0/8 10.0.0.0/8 127.0.0.0/8 169.254.0.0/16 \
172.16.0.0/12 192.0.0.0/24 192.0.2.0/24 224.0.0.0/3 \
192.168.0.0/16 198.18.0.0/15 198.51.100.0/24 \
203.0.113.0/24 }


# Queue List [ Download ]
queue download on cnmac2 bandwidth 70M max 70M
queue media-down parent download bandwidth 20M min 5M max 20M burst 24M for 200ms
queue xbox-down parent media-down bandwidth 4M max 4M burst 8M for 200ms
queue chrome-down parent media-down bandwidth 16M max 16M burst 20M for 225ms
queue std-down parent download bandwidth 50M min 5M max 50M burst 70M for 500ms default


set block-policy drop
set loginterface egress
set skip on lo0
match in all scrub (no-df random-id max-mss 1440)
match out on egress inet from !(egress:network) to any nat-to (egress:0)
block in quick on egress from <martians> to any
block return out quick on egress from any to <martians>
block quick inet6
block all

# A bit of edgy prio and bandwidth queuing, I felt like taking pf out for a test drive here

pass in on $lan_if from $goodguys tag LAN set prio 6
pass in on $wifi_if from $wifiguys tag WIFI modulate state set queue std-down
pass in on $wifi_if from $chromecast tag CHROME modulate state set prio 2 \
set queue chrome-down
block out on $lan_if tagged WIFI
block out on $lan_if tagged CHROME
antispoof for { egress cnmac0 cnmac1 cnmac2 lo0 }
pass in quick on $ext_if from $Jordan to any tag Jordan
block in on $ext_if proto { tcp udp } from any to any port ssh ! tagged Jordan
pass out on $ext_if inet


# Printers Ruleset | Block Printer on Egress && allow $goodguys subnet
block out on $ext_if from $printer to any
pass out quick on $wifi_if from $goodguys to $printer

# Spammers
anchor banlist
load anchor banlist from "/etc/banlist.conf"

# DNS Redirect
anchor dns
load anchor dns from "/etc/dns-redirect.conf"


###

*Anchor banlist.conf:*


# $OpenBSD: pf.conf,v 1.54 2014/08/23 05:49:42 deraadt Exp $
#
## Spammers ##

table <banlist> persist file "/etc/blocklist/banlist.txt"\
file "/etc/blocklist/compromised-ips.txt"\
file "/etc/blocklist/emerging-Block-IPs.txt"\
file "/etc/blocklist/firehol_level3.netset"
block in on egress from <banlist> to any
block out log on egress from any to <banlists>


####

*Anchor dns-redirect.conf:***


# $OpenBSD: pf.conf,v 1.54 2014/08/23 05:49:42 deraadt Exp $
#

wifi_lan="{ cnmac2 }"

# DNS Redirect
pass in on $wifi_lan proto { tcp udp } from any to \
{ 8.8.8.8 8.8.4.4 208.67.222.222 208.67.220.220 209.244.0.3 } port 53 \
tag google rdr-to 172.17.17.1

# I added this because several devices were aggressively pinging 8.8.8.8 on my network and it was annoying me
pass in on $wifi_lan from any to \
{ 8.8.8.8 8.8.4.4 } \
tag google rdr-to 172.17.17.1
Freddy DISSAUX
2017-12-30 08:21:48 UTC
Permalink
Post by Jordan Geoghegan
Hi everyone,
Hello,

[ snip ]
Post by Jordan Geoghegan
cat hosts | grep '^0\.0\.0\.0' | awk '{print "local-zone: \""$2"\" redirect\nlocal-data: \""$2" A 0.0.0.0\""}' > ads.conf
awk 'NF == 2 && $1 == "0.0.0.0" && $2 ~ /^[a-z0-9]/ { print "local-zone: \"" $2 "\" redirect\nlocal-data: \"" $2 " A " $1 "\"" }' host > ads.conf


Regards,
Jordan Geoghegan
2017-12-30 19:36:05 UTC
Permalink
I have tried using all awk for the script before, but I find piping the
grep output into awk to be 2-3x faster on the Edgerouter Lite. I just
ran some timed tests for your script against mine on the ErLite, and I
got similar results, with my script completing in ~6 seconds against the
StevenBlack hosts file, and yours at ~14 seconds. This may not be the
case on more conventional architectures. I am considering rewriting the
script in Perl to see if that runs any faster.
Post by Freddy DISSAUX
Post by Jordan Geoghegan
Hi everyone,
Hello,
[ snip ]
Post by Jordan Geoghegan
cat hosts | grep '^0\.0\.0\.0' | awk '{print "local-zone: \""$2"\" redirect\nlocal-data: \""$2" A 0.0.0.0\""}' > ads.conf
awk 'NF == 2 && $1 == "0.0.0.0" && $2 ~ /^[a-z0-9]/ { print "local-zone: \"" $2 "\" redirect\nlocal-data: \"" $2 " A " $1 "\"" }' host > ads.conf
Regards,
Jordan Geoghegan
2017-12-31 19:29:40 UTC
Permalink
Hi Freddy,

I just ran some further benchmarks between your first and second script,
compared to mine, and again similar results were found. Your second
script was significantly faster than the first, but still didn't match
the grep-piped-into-awk config.

This shouldn't be the case though. I did further testing on my PowerMac
G4 500Mhz workstation running 6.2, which I chose because I thought a
single core ppc G4 500Mhz vs a mips64 dual core 500Mhz would be a pretty
epic showdown. I ran each script twice and wrote the output to /dev/null
to ensure disk I/O wasn't a factor. The StevenBlack hosts file has on
average ~47,000 lines including comments.

The results were somewhat surprising:

The G4 cranked out the scripts with these times:

*Your 1st script: an average of 1.415 seconds**
*

*Your 2nd script: an average of 0.54 seconds*

*My script: an average of 1.71 seconds*


This clearly shows things the way they are supposed to be, with my
script being grossly inefficient and yours being clearly superior. See
below for the times on the Edgerouter Lite:

(Note: All tested times are slower than previous results from last email
due to the machine being under a modest network load during testing.
Load remained consistent due to it being a long running slow 5 megabit
bulk network transfer it was routing. This was unavoidable due to it
being a production machine.)

*Your first script came in at an average of 20.8 seconds*

*Your second script came in at an average of 13.75 seconds *

*And my script came in at an average of 10.25 seconds. *

These results are shockingly poor compared to a G4 of the same clock
speed. The leads me to believe there may be some Octeon specific
inefficiencies at play here, namely floating point. None of the
Edgerouter units have an FPU I believe ( I know for sure the Lite
doesn't) and I am wondering if awk makes heavy use of floating point,
and thus it having to abuse the emulated fpu? During the all awk
scripts, the ERLite becomes cpu bound on 1 core.

It would be awesome if an awk guru here could confirm whether awk makes
heavy use of the fpu.

If this is indeed the case, then the PowerPC would have an extreme
advantage with its beefy AltiVec unit.

So I suppose for those folks running my addblocking scripts, it would be
wise to use Freddy Dissaux's all awk hostfile conversion method if
you're running a more conventional architecture. It would be great if
someone here could post some test results on an arm64 board!

I am now very curious to see how Perl compares against these results. I
hope I can find the time to play around with making a nice optimized
script.
Hello Jordan,
Post by Jordan Geoghegan
I have tried using all awk for the script before, but I find piping the
grep output into awk to be 2-3x faster on the Edgerouter Lite. I just
ran some timed tests for your script against mine on the ErLite, and I
got similar results, with my script completing in ~6 seconds against the
StevenBlack hosts file, and yours at ~14 seconds. This may not be the
case on more conventional architectures. I am considering rewriting the
script in Perl to see if that runs any faster.
Could you try
awk 'BEGIN { OFS = "" } NF == 2 && $1 == "0.0.0.0" { print "local-zone: \"", $2, "\" redirect"; print "local-data: \"", $2, " A 0.0.0.0\"" }' hosts > ads.conf
If i understand my tests, 2 print without concat are faster than
1 print with concat (and faster than 1 printf)
Post by Jordan Geoghegan
Post by Jordan Geoghegan
cat hosts | grep '^0\.0\.0\.0' | awk '{print "local-zone: \""$2"\" redirect\nlocal-data: \""$2" A 0.0.0.0\""}' > ads.conf
grep '^0\.0\.0\.0' host | awk '{print "local-zone: \""$2"\" redirect\nlocal-data: \""$2" A 0.0.0.0\""}' > ads.conf
Regards,
jin&hitman&Barracuda
2018-08-24 13:32:39 UTC
Permalink
Hello

Thanks for sharing all those informations. I've been looking a way to
create a blacklist and you sent this mail just on time. Your web page help
me a lot.
On the OpenBSD your script do all jobs but on linux based systems I wrote a
shell script for update iptables rules.

http://analog-radyo.blogspot.com/2018/08/dynamic-block-list-on-linux-iptables.html
Post by Jordan Geoghegan
Hi everyone,
Due to the number of people who have requested my add-blocking scripts,
network-wide bad-host/add-blocking.
I won't go into detail on how to set up routing/dhcp/unbound/anchors
etc, for that see: https://www.openbsd.org/faq/pf/example1.html
I've included some example files from my an Edgerouter I have set up .
They are trimmed down for brevities sake; the conf files are not
production ready, these are merely examples.
This setup is easily customizable, if you come across any other block
lists you prefer, then they can be dropped in no problem. I chose to use
solely the StevenBlack hosts file because it is a master list compiled
from all the major banlists found in popular blocking products such as
uBlock Origin, Addblock Plus et al. I also chose this file because it is
filtered for duplicates as unbound(8) is said to struggle when there are
redundancies in the blocklists, I'm told -- though I've never had any issue.
You're going to have to read the scripts and create the directories the
scripts are calling and edit the anchor macros to fit your interface
layout (I doubt everyone here is running cnmac0 as egress) and also will
have to make the scripts executable and set them to run at regular
intervals with crontab, ideally nightly.
I didn't make these scripts intelligent because I figured it was simpler
to just run mkdir once rather than add extra lines to the script.
I know the pf.conf is fairly long, I thought I would show an example of
my prio and queing setup as an example, or conversely to see if anyone
can poke any holes in it.
All the relevant bits regarding the anchors and blocklists are found at
the end of the pf.conf file. See below that for the anchor conf files
we're calling as well.
Hope this helps,
Jordan Geoghegan
*DNS addblock script:*
cd /var/unbound/etc/banlist && \
ftp https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts && \
cat hosts | grep '^0\.0\.0\.0' | awk '{print "local-zone: \""$2"\"
redirect\nlocal-data: \""$2" A 0.0.0.0\""}' > ads.conf
rcctl reload unbound
###
*IP based malicious IP blocking:*
cd /etc/blocklist && ftp https://www.binarydefense.com/banlist.txt\
&& <https://www.binarydefense.com/banlist.txt%5C&&> ftp
https://rules.emergingthreats.net/blockrules/compromised-ips.txt\
&& <https://rules.emergingthreats.net/blockrules/compromised-ips.txt%5C&&>
ftp https://rules.emergingthreats.net/fwrules/emerging-Block-IPs.txt\
&& <https://rules.emergingthreats.net/fwrules/emerging-Block-IPs.txt%5C&&>
ftp
https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/firehol_level3.netset\
&&
<https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/firehol_level3.netset%5C&&>
pfctl -a banlist -f /etc/banlist.conf
###
As you can see, we are going to have to make an anchor in pf called
'banlist' and modify the unbound.conf to load our banlist 'ads.conf'
If that's all you need, then you're pretty much good to go. If you would
like to see my example conf files, see below.
*
Example unbound.conf:*
# $OpenBSD: unbound.conf,v 1.7 2016/03/30 01:41:25 sthen Exp $
interface: 172.17.17.1
interface: 127.0.0.1
access-control: 172.17.17.0/24 allow
access-control: 172.17.0.0/24 allow
do-not-query-localhost: no
hide-identity: yes
hide-version: yes
include: /var/unbound/etc/banlist/ads.conf
name: "."
forward-addr: UR.DNS.GO.HERE
forward-addr: UR.DNS.GO.HERE
###
*Example pf.conf:*
# $OpenBSD: pf.conf,v 1.54 2014/08/23 05:49:42 deraadt Exp $
#
# See pf.conf(5) and /etc/examples/pf.conf
# By default, do not permit remote connections to X11
block return in on ! lo0 proto tcp to port 6000:6010
#
ext_if="{ cnmac0 }"
int_if="{ cnmac1 cnmac2 }"
lan_if="{ cnmac1 }"
wifi_if="{ cnmac2 }"
goodguys="{ 172.17.17.0/24 }"
wifiguys="{ 172.17.0.0/24 }"
chromecast="{ 172.17.0.12 172.17.0.13 172.17.0.23 }"
xbox360="{ 172.17.0.19 }"
printer="{ 172.17.0.17 }"
Jordan="{ XXX.XX.XXX.XX }"
table <martians> { 0.0.0.0/8 10.0.0.0/8 127.0.0.0/8 169.254.0.0/16 \
172.16.0.0/12 192.0.0.0/24 192.0.2.0/24 224.0.0.0/3 \
192.168.0.0/16 198.18.0.0/15 198.51.100.0/24 \
203.0.113.0/24 }
# Queue List [ Download ]
queue download on cnmac2 bandwidth 70M max 70M
queue media-down parent download bandwidth 20M min 5M max 20M burst 24M for 200ms
queue xbox-down parent media-down bandwidth 4M max 4M burst 8M for 200ms
queue chrome-down parent media-down bandwidth 16M max 16M burst 20M for 225ms
queue std-down parent download bandwidth 50M min 5M max 50M burst 70M for 500ms default
set block-policy drop
set loginterface egress
set skip on lo0
match in all scrub (no-df random-id max-mss 1440)
match out on egress inet from !(egress:network) to any nat-to (egress:0)
block in quick on egress from <martians> to any
block return out quick on egress from any to <martians>
block quick inet6
block all
# A bit of edgy prio and bandwidth queuing, I felt like taking pf out for a test drive here
pass in on $lan_if from $goodguys tag LAN set prio 6
pass in on $wifi_if from $wifiguys tag WIFI modulate state set queue std-down
pass in on $wifi_if from $chromecast tag CHROME modulate state set prio 2 \
set queue chrome-down
block out on $lan_if tagged WIFI
block out on $lan_if tagged CHROME
antispoof for { egress cnmac0 cnmac1 cnmac2 lo0 }
pass in quick on $ext_if from $Jordan to any tag Jordan
block in on $ext_if proto { tcp udp } from any to any port ssh ! tagged Jordan
pass out on $ext_if inet
# Printers Ruleset | Block Printer on Egress && allow $goodguys subnet
block out on $ext_if from $printer to any
pass out quick on $wifi_if from $goodguys to $printer
# Spammers
anchor banlist
load anchor banlist from "/etc/banlist.conf"
# DNS Redirect
anchor dns
load anchor dns from "/etc/dns-redirect.conf"
###
*Anchor banlist.conf:*
# $OpenBSD: pf.conf,v 1.54 2014/08/23 05:49:42 deraadt Exp $
#
## Spammers ##
table <banlist> persist file "/etc/blocklist/banlist.txt"\
file "/etc/blocklist/compromised-ips.txt"\
file "/etc/blocklist/emerging-Block-IPs.txt"\
file "/etc/blocklist/firehol_level3.netset"
block in on egress from <banlist> to any
block out log on egress from any to <banlists>
####
*Anchor dns-redirect.conf:***
# $OpenBSD: pf.conf,v 1.54 2014/08/23 05:49:42 deraadt Exp $
#
wifi_lan="{ cnmac2 }"
# DNS Redirect
pass in on $wifi_lan proto { tcp udp } from any to \
{ 8.8.8.8 8.8.4.4 208.67.222.222 208.67.220.220 209.244.0.3 } port 53 \
tag google rdr-to 172.17.17.1
# I added this because several devices were aggressively pinging 8.8.8.8
on my network and it was annoying me
pass in on $wifi_lan from any to \
{ 8.8.8.8 8.8.4.4 } \
tag google rdr-to 172.17.17.1
--
*There is no place like "/home"*
*Tuco (Benedicto Pasifico Juan Maria) Ramirez*
Jordan Geoghegan
2018-08-25 07:31:22 UTC
Permalink
You may want to check out the more recent guides I wrote for the updated
version of these scripts:

www.geoghegan.ca/unbound-adblock.html

www.geoghegan.ca/pfbadhost.html
Post by jin&hitman&Barracuda
Hello
Thanks for sharing all those informations. I've been looking a way to
create a blacklist and you sent this mail just on time. Your web page
help me a lot.
On the OpenBSD your script do all jobs but on linux based systems I
wrote a shell script for update iptables rules.
http://analog-radyo.blogspot.com/2018/08/dynamic-block-list-on-linux-iptables.html
Hi everyone,
Due to the number of people who have requested my add-blocking scripts,
network-wide bad-host/add-blocking.
I won't go into detail on how to set up routing/dhcp/unbound/anchors
etc, for that see: https://www.openbsd.org/faq/pf/example1.html
I've included some example files from my an Edgerouter I have set up .
They are trimmed down for brevities sake; the conf files are not
production ready, these are merely examples.
This setup is easily customizable, if you come across any other block
lists you prefer, then they can be dropped in no problem. I chose to use
solely the StevenBlack hosts file because it is a master list compiled
from all the major banlists found in popular blocking products such as
uBlock Origin, Addblock Plus et al. I also chose this file because it is
filtered for duplicates as unbound(8) is said to struggle when there are
redundancies in the blocklists, I'm told -- though I've never had any issue.
You're going to have to read the scripts and create the
directories the
scripts are calling and edit the anchor macros to fit your interface
layout (I doubt everyone here is running cnmac0 as egress) and also will
have to make the scripts executable and set them to run at regular
intervals with crontab, ideally nightly.
I didn't make these scripts intelligent because I figured it was simpler
to just run mkdir once rather than add extra lines to the script.
I know the pf.conf is fairly long, I thought I would show an example of
my prio and queing setup as an example, or conversely to see if anyone
can poke any holes in it.
All the relevant bits regarding the anchors and blocklists are found at
the end of the pf.conf file. See below that for the anchor conf files
we're calling as well.
Hope this helps,
Jordan Geoghegan
*DNS addblock script:*
cd /var/unbound/etc/banlist && \
ftp
https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts && \
cat hosts | grep '^0\.0\.0\.0' | awk '{print "local-zone: \""$2"\"
redirect\nlocal-data: \""$2" A 0.0.0.0\""}' > ads.conf
rcctl reload unbound
###
*IP based malicious IP blocking:*
cd /etc/blocklist && ftp https://www.binarydefense.com/banlist.txt\
&& <https://www.binarydefense.com/banlist.txt%5C&&> ftp
https://rules.emergingthreats.net/blockrules/compromised-ips.txt\
&&
<https://rules.emergingthreats.net/blockrules/compromised-ips.txt%5C&&>
ftp https://rules.emergingthreats.net/fwrules/emerging-Block-IPs.txt\
&&
<https://rules.emergingthreats.net/fwrules/emerging-Block-IPs.txt%5C&&>
ftp
https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/firehol_level3.netset\
&&
<https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/firehol_level3.netset%5C&&>
pfctl -a banlist -f /etc/banlist.conf
###
As you can see, we are going to have to make an anchor in pf called
'banlist' and modify the unbound.conf to load our banlist 'ads.conf'
If that's all you need, then you're pretty much good to go. If you would
like to see my example conf files, see below.
*
Example unbound.conf:*
# $OpenBSD: unbound.conf,v 1.7 2016/03/30 01:41:25 sthen Exp $
        interface: 172.17.17.1
        interface: 127.0.0.1
        access-control: 172.17.17.0/24 <http://172.17.17.0/24> allow
        access-control: 172.17.0.0/24 <http://172.17.0.0/24> allow
        do-not-query-localhost: no
        hide-identity: yes
        hide-version: yes
        include: /var/unbound/etc/banlist/ads.conf
        name: "."
        forward-addr: UR.DNS.GO.HERE
        forward-addr: UR.DNS.GO.HERE
###
*Example pf.conf:*
#       $OpenBSD: pf.conf,v 1.54 2014/08/23 05:49:42 deraadt Exp $
#
# See pf.conf(5) and /etc/examples/pf.conf
# By default, do not permit remote connections to X11
block return in on ! lo0 proto tcp to port 6000:6010
#
ext_if="{ cnmac0 }"
int_if="{ cnmac1 cnmac2 }"
lan_if="{ cnmac1 }"
wifi_if="{ cnmac2 }"
goodguys="{ 172.17.17.0/24 <http://172.17.17.0/24> }"
wifiguys="{ 172.17.0.0/24 <http://172.17.0.0/24> }"
chromecast="{ 172.17.0.12 172.17.0.13 172.17.0.23 }"
xbox360="{ 172.17.0.19 }"
printer="{ 172.17.0.17 }"
Jordan="{ XXX.XX.XXX.XX }"
table <martians> { 0.0.0.0/8 <http://0.0.0.0/8> 10.0.0.0/8
<http://10.0.0.0/8> 127.0.0.0/8 <http://127.0.0.0/8>
169.254.0.0/16 <http://169.254.0.0/16>     \
172.16.0.0/12 <http://172.16.0.0/12> 192.0.0.0/24
<http://192.0.0.0/24> 192.0.2.0/24 <http://192.0.2.0/24>
224.0.0.0/3 <http://224.0.0.0/3>  \
192.168.0.0/16 <http://192.168.0.0/16> 198.18.0.0/15
<http://198.18.0.0/15> 198.51.100.0/24 <http://198.51.100.0/24> \
203.0.113.0/24 <http://203.0.113.0/24> }
# Queue List [ Download ]
queue download on cnmac2 bandwidth 70M max 70M
queue media-down parent download bandwidth 20M min 5M max 20M burst 24M for 200ms
queue xbox-down parent media-down bandwidth 4M max 4M burst 8M for 200ms
queue chrome-down parent media-down bandwidth 16M max 16M burst 20M for 225ms
queue std-down parent download bandwidth 50M min 5M max 50M burst
70M for 500ms default
set block-policy drop
set loginterface egress
set skip on lo0
match in all scrub (no-df random-id max-mss 1440)
match out on egress inet from !(egress:network) to any nat-to (egress:0)
block in quick on egress from <martians> to any
block return out quick on egress from any to <martians>
block quick inet6
block all
# A bit of edgy prio and bandwidth queuing, I felt like taking pf
out for a test drive here
pass in on $lan_if from $goodguys tag LAN set prio 6
pass in on $wifi_if from $wifiguys tag WIFI modulate state set queue std-down
pass in on $wifi_if from $chromecast tag CHROME modulate state set prio 2 \
set queue chrome-down
block out on $lan_if tagged WIFI
block out on $lan_if tagged CHROME
antispoof for { egress cnmac0 cnmac1 cnmac2 lo0 }
pass in quick on $ext_if from $Jordan to any tag Jordan
block in on $ext_if proto { tcp udp } from any to any port ssh ! tagged Jordan
pass out on $ext_if inet
# Printers Ruleset      | Block Printer on Egress && allow
$goodguys subnet
block out on $ext_if from $printer to any
pass out quick on $wifi_if from $goodguys to $printer
# Spammers
anchor banlist
load anchor banlist from "/etc/banlist.conf"
# DNS Redirect
anchor dns
load anchor dns from "/etc/dns-redirect.conf"
###
*Anchor banlist.conf:*
#   $OpenBSD: pf.conf,v 1.54 2014/08/23 05:49:42 deraadt Exp $
#
## Spammers ##
table <banlist> persist file "/etc/blocklist/banlist.txt"\
file "/etc/blocklist/compromised-ips.txt"\
file "/etc/blocklist/emerging-Block-IPs.txt"\
file "/etc/blocklist/firehol_level3.netset"
block in on egress from <banlist> to any
block out log on egress from any to <banlists>
####
*Anchor  dns-redirect.conf:***
#   $OpenBSD: pf.conf,v 1.54 2014/08/23 05:49:42 deraadt Exp $
#
wifi_lan="{ cnmac2 }"
# DNS Redirect
pass in on $wifi_lan proto { tcp udp } from any to \
{ 8.8.8.8 8.8.4.4 208.67.222.222 208.67.220.220 209.244.0.3 } port 53 \
tag google rdr-to 172.17.17.1
# I added this because several devices were aggressively pinging
8.8.8.8 on my network and it was annoying me
pass in on $wifi_lan from any to \
{ 8.8.8.8 8.8.4.4  } \
tag google rdr-to 172.17.17.1
--
/There is no place like "/home"/
/Tuco (Benedicto Pasifico Juan Maria) Ramirez/
jin&hitman&Barracuda
2018-08-25 09:33:13 UTC
Permalink
Thanks Jordan, i will look at those links.
Post by Jordan Geoghegan
You may want to check out the more recent guides I wrote for the updated
www.geoghegan.ca/unbound-adblock.html
www.geoghegan.ca/pfbadhost.html
Post by jin&hitman&Barracuda
Hello
Thanks for sharing all those informations. I've been looking a way to
create a blacklist and you sent this mail just on time. Your web page
help me a lot.
On the OpenBSD your script do all jobs but on linux based systems I
wrote a shell script for update iptables rules.
http://analog-radyo.blogspot.com/2018/08/dynamic-block-list-on-linux-iptables.html
Post by jin&hitman&Barracuda
Hi everyone,
Due to the number of people who have requested my add-blocking scripts,
network-wide bad-host/add-blocking.
I won't go into detail on how to set up routing/dhcp/unbound/anchors
etc, for that see: https://www.openbsd.org/faq/pf/example1.html
I've included some example files from my an Edgerouter I have set up .
They are trimmed down for brevities sake; the conf files are not
production ready, these are merely examples.
This setup is easily customizable, if you come across any other block
lists you prefer, then they can be dropped in no problem. I chose to use
solely the StevenBlack hosts file because it is a master list compiled
from all the major banlists found in popular blocking products such as
uBlock Origin, Addblock Plus et al. I also chose this file because it is
filtered for duplicates as unbound(8) is said to struggle when there are
redundancies in the blocklists, I'm told -- though I've never had any issue.
You're going to have to read the scripts and create the
directories the
scripts are calling and edit the anchor macros to fit your interface
layout (I doubt everyone here is running cnmac0 as egress) and also will
have to make the scripts executable and set them to run at regular
intervals with crontab, ideally nightly.
I didn't make these scripts intelligent because I figured it was simpler
to just run mkdir once rather than add extra lines to the script.
I know the pf.conf is fairly long, I thought I would show an example of
my prio and queing setup as an example, or conversely to see if anyone
can poke any holes in it.
All the relevant bits regarding the anchors and blocklists are found at
the end of the pf.conf file. See below that for the anchor conf files
we're calling as well.
Hope this helps,
Jordan Geoghegan
*DNS addblock script:*
cd /var/unbound/etc/banlist && \
ftp
https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts &&
\
Post by jin&hitman&Barracuda
cat hosts | grep '^0\.0\.0\.0' | awk '{print "local-zone: \""$2"\"
redirect\nlocal-data: \""$2" A 0.0.0.0\""}' > ads.conf
rcctl reload unbound
###
*IP based malicious IP blocking:*
cd /etc/blocklist && ftp https://www.binarydefense.com/banlist.txt\
&& <https://www.binarydefense.com/banlist.txt%5C&&> ftp
https://rules.emergingthreats.net/blockrules/compromised-ips.txt\
&&
<
https://rules.emergingthreats.net/blockrules/compromised-ips.txt%5C&&>
Post by jin&hitman&Barracuda
ftp
https://rules.emergingthreats.net/fwrules/emerging-Block-IPs.txt\
Post by jin&hitman&Barracuda
&&
<
https://rules.emergingthreats.net/fwrules/emerging-Block-IPs.txt%5C&&>
Post by jin&hitman&Barracuda
ftp
https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/firehol_level3.netset\
Post by jin&hitman&Barracuda
&&
<
https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/firehol_level3.netset%5C&&
Post by jin&hitman&Barracuda
pfctl -a banlist -f /etc/banlist.conf
###
As you can see, we are going to have to make an anchor in pf called
'banlist' and modify the unbound.conf to load our banlist 'ads.conf'
If that's all you need, then you're pretty much good to go. If you would
like to see my example conf files, see below.
*
Example unbound.conf:*
# $OpenBSD: unbound.conf,v 1.7 2016/03/30 01:41:25 sthen Exp $
interface: 172.17.17.1
interface: 127.0.0.1
access-control: 172.17.17.0/24 <http://172.17.17.0/24> allow
access-control: 172.17.0.0/24 <http://172.17.0.0/24> allow
do-not-query-localhost: no
hide-identity: yes
hide-version: yes
include: /var/unbound/etc/banlist/ads.conf
name: "."
forward-addr: UR.DNS.GO.HERE
forward-addr: UR.DNS.GO.HERE
###
*Example pf.conf:*
# $OpenBSD: pf.conf,v 1.54 2014/08/23 05:49:42 deraadt Exp $
#
# See pf.conf(5) and /etc/examples/pf.conf
# By default, do not permit remote connections to X11
block return in on ! lo0 proto tcp to port 6000:6010
#
ext_if="{ cnmac0 }"
int_if="{ cnmac1 cnmac2 }"
lan_if="{ cnmac1 }"
wifi_if="{ cnmac2 }"
goodguys="{ 172.17.17.0/24 <http://172.17.17.0/24> }"
wifiguys="{ 172.17.0.0/24 <http://172.17.0.0/24> }"
chromecast="{ 172.17.0.12 172.17.0.13 172.17.0.23 }"
xbox360="{ 172.17.0.19 }"
printer="{ 172.17.0.17 }"
Jordan="{ XXX.XX.XXX.XX }"
table <martians> { 0.0.0.0/8 <http://0.0.0.0/8> 10.0.0.0/8
<http://10.0.0.0/8> 127.0.0.0/8 <http://127.0.0.0/8>
169.254.0.0/16 <http://169.254.0.0/16> \
172.16.0.0/12 <http://172.16.0.0/12> 192.0.0.0/24
<http://192.0.0.0/24> 192.0.2.0/24 <http://192.0.2.0/24>
224.0.0.0/3 <http://224.0.0.0/3> \
192.168.0.0/16 <http://192.168.0.0/16> 198.18.0.0/15
<http://198.18.0.0/15> 198.51.100.0/24 <http://198.51.100.0/24> \
203.0.113.0/24 <http://203.0.113.0/24> }
# Queue List [ Download ]
queue download on cnmac2 bandwidth 70M max 70M
queue media-down parent download bandwidth 20M min 5M max 20M
burst 24M for 200ms
queue xbox-down parent media-down bandwidth 4M max 4M burst 8M for 200ms
queue chrome-down parent media-down bandwidth 16M max 16M burst 20M for 225ms
queue std-down parent download bandwidth 50M min 5M max 50M burst
70M for 500ms default
set block-policy drop
set loginterface egress
set skip on lo0
match in all scrub (no-df random-id max-mss 1440)
match out on egress inet from !(egress:network) to any nat-to (egress:0)
block in quick on egress from <martians> to any
block return out quick on egress from any to <martians>
block quick inet6
block all
# A bit of edgy prio and bandwidth queuing, I felt like taking pf
out for a test drive here
pass in on $lan_if from $goodguys tag LAN set prio 6
pass in on $wifi_if from $wifiguys tag WIFI modulate state set queue std-down
pass in on $wifi_if from $chromecast tag CHROME modulate state set prio 2 \
set queue chrome-down
block out on $lan_if tagged WIFI
block out on $lan_if tagged CHROME
antispoof for { egress cnmac0 cnmac1 cnmac2 lo0 }
pass in quick on $ext_if from $Jordan to any tag Jordan
block in on $ext_if proto { tcp udp } from any to any port ssh ! tagged Jordan
pass out on $ext_if inet
# Printers Ruleset | Block Printer on Egress && allow $goodguys subnet
block out on $ext_if from $printer to any
pass out quick on $wifi_if from $goodguys to $printer
# Spammers
anchor banlist
load anchor banlist from "/etc/banlist.conf"
# DNS Redirect
anchor dns
load anchor dns from "/etc/dns-redirect.conf"
###
*Anchor banlist.conf:*
# $OpenBSD: pf.conf,v 1.54 2014/08/23 05:49:42 deraadt Exp $
#
## Spammers ##
table <banlist> persist file "/etc/blocklist/banlist.txt"\
file "/etc/blocklist/compromised-ips.txt"\
file "/etc/blocklist/emerging-Block-IPs.txt"\
file "/etc/blocklist/firehol_level3.netset"
block in on egress from <banlist> to any
block out log on egress from any to <banlists>
####
*Anchor dns-redirect.conf:***
# $OpenBSD: pf.conf,v 1.54 2014/08/23 05:49:42 deraadt Exp $
#
wifi_lan="{ cnmac2 }"
# DNS Redirect
pass in on $wifi_lan proto { tcp udp } from any to \
{ 8.8.8.8 8.8.4.4 208.67.222.222 208.67.220.220 209.244.0.3 } port 53 \
tag google rdr-to 172.17.17.1
# I added this because several devices were aggressively pinging
8.8.8.8 on my network and it was annoying me
pass in on $wifi_lan from any to \
{ 8.8.8.8 8.8.4.4 } \
tag google rdr-to 172.17.17.1
--
/There is no place like "/home"/
/Tuco (Benedicto Pasifico Juan Maria) Ramirez/
Loading...