################################################################### # pf-badhost 0.5 FreeBSD Installation Instructions # Copyright 2018-2021 Jordan Geoghegan ################################################################### ################################################################### # Table of Contents: ################################################################### * Upgrade Instructions (upgrade from 0.4) * Fresh Install Instructions * Post Install Notes ################################################################### # Upgrade from pf-badhost 0.4 ################################################################### 1) Download updated script: $ fetch https://geoghegan.ca/pub/pf-badhost/0.5/pf-badhost.sh 2) Install script with appropriate permissions and remove old install location: # install -m 755 -o root -g bin pf-badhost.sh /usr/local/bin/pf-badhost # rm /usr/local/bin/pf-badhost.sh 3) Update _pfbadhost users home folder: Previous pf-badhost releases incorrectly set the users home folder # pw usermod _pfbadhost -d /var/empty 4) Update doas.conf - pf-badhost now uses absolute paths in doas.conf $ cat /usr/local/etc/doas.conf ... permit root permit nopass _pfbadhost cmd /sbin/pfctl args -nf /etc/pf.conf permit nopass _pfbadhost cmd /sbin/pfctl args -t pfbadhost -T replace -f /etc/pf-badhost.txt # Optional rule for authlog scanning permit nopass _pfbadhost cmd /usr/bin/bzcat args -f /var/log/authlog /var/log/authlog.0.gz ... 5) Create pf-badhost log directory: # install -d -o root -g wheel -m 755 /var/log/pf-badhost # install -o _pfbadhost -g wheel -m 640 /dev/null /var/log/pf-badhost/pf-badhost.log # install -o _pfbadhost -g wheel -m 640 /dev/null /var/log/pf-badhost/pf-badhost.log.0.gz 6) Update cron job (See 'Post Install Notes' below for more info): # crontab -u _pfbadhost -e ... SHELL=/bin/sh @daily sleep $(echo $((RANDOM\%7200+1))) ; pf-badhost -O freebsd ... 7) Install ksh (See 'Post Install Notes' below for more info): # pkg install oksh # ln -s /usr/local/bin/oksh /usr/local/bin/ksh 8) pf-badhost now automagically detects if RipGrep is installed. If you were using it with previous versions of pf-badhost, then no further action is required. --- Similarly, pf-badhost also automagically detects if mawk or gawk is present. This can improve performance on low powered devices significantly. 9) To enable additional features such as IPv6, Subnet Aggregation, Geo-Blocking, Bogon Filtering or Authlog Scanning open "/usr/local/bin/pf-badhost" with your text editor of choice and find the "User Configuration Area" near the top of the file where you can enable features by setting their value to "1" --- Most options can now be configured via command line flags. See man page for more details. --- See the "Notes" section below for more info on installing optional utilities. Note: If you have upgraded to FreeBSD 12.2, you will need to move the 'net.pf.request_maxcount=400000' line from '/boot/loader.conf' to '/etc/sysctl' as this is now a runtime tunable option. See here for more info: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=250780 Please read the man page for information on how to configure pf-badhost. The manpage can be found here: https://www.geoghegan.ca/pub/pf-badhost/0.5/man/man.txt To receive notification of new pf-badhost releases and updates please send an email to 'announce@geoghegan.ca' with a subject line and body of "subscribe pf-badhost" ################################################################### # Fresh Installation Guide ################################################################### 1) Create a new user (we'll call ours "_pfbadhost"): The user should be created with a default shell of "nologin", home folder set to /var/empty/ with no password specified (disables password logins) # pw useradd _pfbadhost -s /sbin/nologin -d /var/empty 2) Download script: $ fetch https://geoghegan.ca/pub/pf-badhost/0.5/pf-badhost.sh 3) Install script with appropriate permissions: # install -m 755 -o root -g bin pf-badhost.sh /usr/local/bin/pf-badhost 4) Create required files: # install -m 640 -o _pfbadhost -g wheel /dev/null /etc/pf-badhost.txt # install -d -m 755 -o root -g wheel /var/log/pf-badhost # install -m 640 -o _pfbadhost -g wheel /dev/null /var/log/pf-badhost/pf-badhost.log # install -m 640 -o _pfbadhost -g wheel /dev/null /var/log/pf-badhost/pf-badhost.log.0.gz 5.a) Install "doas" utility and ksh: # pkg install doas oksh # ln -s /usr/local/bin/oksh /usr/local/bin/ksh 5.b) OPTIONAL: Install mawk for improved performance: # pkg install mawk 6) Give user "_pfbadhost" strict doas permission for the exact commands the script needs run as superuser. NOTE: Unlike "sudo", _ALL_ users must be explicitly granted permission to use doas, even the root user. $ cat /usr/local/etc/doas.conf ... permit root permit nopass _pfbadhost cmd /sbin/pfctl args -nf /etc/pf.conf permit nopass _pfbadhost cmd /sbin/pfctl args -t pfbadhost -T replace -f /etc/pf-badhost.txt # Optional rule for authlog scanning permit nopass _pfbadhost cmd /usr/bin/bzcat args -f /var/log/auth.log /var/log/auth.log.0.bz2 ... 7) Add the following lines to your pf.conf: (Putting it higher-up/earlier in the ruleset is recommended) ... set limit table-entries 400000 table persist file "/etc/pf-badhost.txt" block in quick on egress from block out quick on egress to ... 8.a) Increase pfctl request/table size limits: NOTE: If using FreeBSD version 12.1 or older, skip to the next step. # cat /etc/sysctl.conf ... net.pf.request_maxcount=400000 ... -- Use sysctl to make runtime change to avoid needing to reboot: # sysctl net.pf.request_maxcount=400000 8.b) NOTE: Only run this step If using FreeBSD version 12.1 or older -- Add add a line to '/boot/loader.conf', as prior to 12.2, this was not configurable at runtime. # cat /boot/loader.conf ... net.pf.request_maxcount=400000 ... -- Then reboot: # shutdown -r now 9) To enable additional features such as IPv6, Subnet Aggregation, Geo-Blocking, Bogon Filtering or Authlog Scanning open "/usr/local/bin/pf-badhost" with your text editor of choice and find the "User Configuration Area" near the top of the file where you can enable features by setting their value to "1" --- Most options can also be configured via command line flags. See man page for more details. --- See the "Notes" section below for more info on installing optional utilities. 10) Run pfbadhost as user "_pfbadhost" using the "-O freebsd" argument: # doas -u _pfbadhost pf-badhost -O freebsd 11) Reload your pf rule set: # pfctl -f /etc/pf.conf 12) For good measure, we'll run pf-badhost once more: # doas -u _pfbadhost pf-badhost -O freebsd 13) Edit _pfbadhost crontab to run pf-badhost every night at midnight: # crontab -u _pfbadhost -e ... SHELL=/bin/sh @daily sleep $(echo $((RANDOM\%7200+1))) ; pf-badhost -O freebsd ... Yay! pf-badhost is now installed! With the nightly cron job, the list will be regularly updated with the latest known bad hosts. Please read the man page for information on how to configure pf-badhost. The manpage can be found here: https://www.geoghegan.ca/pub/pf-badhost/0.5/man/man.txt To receive notification of new pf-badhost releases and updates please send an email to 'announce@geoghegan.ca' with a subject line and body of "subscribe pf-badhost" ################################################################### # Post Install Notes: ################################################################### Note: For info on enabling PF on FreeBSD see: https://www.freebsd.org/doc/handbook/firewalls-pf.html Note: FreeBSD 12.2 shipped with a major foot-gun related to "net.pf.request_maxcount" being moved from a "tunable" set within "/boot/loader.conf" into a runtime option set within "/etc/sysctl.conf" See: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=250780 X) To add custom rules or enable features, or add alternate blocklists, See the "User Configuration Area" located at the top of the script. This area serves as a built in config file, so please feel free to edit it and experiment with all the features available within. --- Note: Most options can also be configured from the command line X) Regarding Shells: pf-badhost requires a shell that supports 'typeset' and ksh array syntanx. FreeBSD's default /bin/sh doesn't behave the way we need it to, so a modern shell should be installed. Several shells in FreeBSD's package repository have been tested and confirmed to work: * oksh (OpenBSD's ksh) * ksh93 * zsh * bash ... If you wish to use one of the above shells, you'll have to make a few changes. To make these changes automatically (replace "zsh" with your preferred shell): # sed -i -e '1 s/ksh/zsh/' /usr/local/bin/pf-badhost X) Regarding Cron Jobs: Over the past year I've noticed a number of list host servers going down at midnight in populated timezones (ie West Coast, East Coast and Western Europe). To be respectful (and to avoid overloading list providers servers) we have cron jobs scheduled to run at a random time within a defined interval. --- With the new default cron job, pf-badhost will be run every night at some point between midnight and 2AM, and thus distributing the load of thousands of queries from numerous users over a 2 hour period rather than a matter of seconds. 1) The script is able to detect which (if any) subnet aggregation utilities are installed and will try to "Do The Right Thing(tm)" and fallback to the best available option. If no subnet aggregation utility is found, the script will fallback to using a pure Perl IPv4 aggregator if Perl is installed. Despite its name, "aggregate6" supports both IPv4 and IPv6 addresses and is written in Python, whereas the "aggregate" utility supports only IPv4 addresses and is written in C and uses significantly less memory but runs much slower. For greatly improved performance, aggregate6 can be run with Pypy. If both utilities are installed, the C based "aggregate" utility will be preferred for IPv4 aggregation, but the script will happily function if only one or the other is installed (or neither). --- Note: Subnet aggregation can be enabled with the '-A' switch on the commandline. * "aggregate" can be installed via: # pkg install aggregate * "aggregate6" can be installed via: # pip install aggregate6 * For greatly improved aggregation performance, run aggregate6 with Pypy: # pkg install pypy3 # pypy3 -m ensurepip # pypy3 -m pip install aggregate6 // Note the Pypy version number, yours may be different! # ln -s /usr/local/pypy3-7.3/bin/aggregate6 /usr/local/bin/aggregate6 * The experimental aggregator "aggy" can be installed like so: ... # pkg install go $ fetch https://geoghegan.ca/pub/aggy/0.1/aggy.go $ go build aggy.go # install -m 755 -o root -g bin aggy /usr/local/bin/aggy ... 2) If you would like to update pf-badhost blocklists at a more frequent interval you can use a modified cron job: --- It is essential that you make use of the crontab randomization feature to ensure use of pf-badhost remains respecful of blocklist provider resources. --- The following example runs pf-badhost every 3 hours at a random minute (this avoids flooding the server with traffic at exactly XX:00) --- Example: Run pf-badhost every 3 to 4 hours # crontab -u _pfbadhost -e ... 0 */3 * * * sleep $(echo $((RANDOM\%3600+1))) ; pf-badhost -O freebsd ...