I've replaced my ISP's router with an open source, ARM64-based device from Traverse Technologies. It uses an NXP LS1088A, an 8-core variant of the CPU which was at one point offered as part of a module for the MNT Reform. While it can handle up to 32GB of DDR4 RAM, I've only got 16GB installed in mine. This is much, much more than a simple router needs; the device is designed to double as a home server, and I intend to eventually expand it into such.
This device's configuration files, as they deviate from the defaults, are reproduced here, but this write-up is still a work-in-progress. I'll expand on my explanations as I have time and inclination, as is generally the case in this digital garden of mine.
I've removed all sensitive information in the files below and replaced it with ALL CAPS placeholders rendered between angle brackets. Hopefully these placeholders are self-descriptive, but in case they aren't, I've provided a "placeholder legend" near the bottom of the page.
I am going off of my bash history and my memory here, so it's entirely possible I've missed something. Please send an email to my pseudonym at my domain if, in following my footsteps, you find I have.
$ sudo apt install netplan.io wireguard bind9 pppoeconf firewalld kea-dhcp4-server
Why did I choose these packages?
firewalld try to automatically rollback changes if they broke my Internet connection was very appealing. I also liked the idea of defining firewall rules via XML files rather than through commands in a shell script that I would have to sequence just so. E.g., my home server uses a shell script with "up" and "down" functions which contain iptables -A in the former and the inverse iptables -D commands in the latter. When I add a new rule, I have to run "down" without the removal command for the new rule, run "up" with the new rule, and then add the removal rule. It's a bit tedious and error-prone. Firewalld looked like an appealing alternative.The first step in building a Linux router is turning on IPv4 forwarding:
net.ipv4.ip_forward=1
/etc/sysctl.conf
This next configuration was harder to arrive at than it should have been, primarily because I did not properly understand the difference between bridging and bonding, despite (likely) having learned about it at least once or twice before, and despite having certainly used bridging before when I build an OpenBSD-based router years ago. I blindly worked from the example given in Netplan's documentation which presents a device configured with multiple bonds as "acting as a router." I omitted the "mode" portion at first and found everything worked, but this was likely because the default bonding algorithm is "round robin," which means all connected devices were likely eventually getting the packets directed to them as they got retransmitted after the other end failed to receive an "ACK." Adding the "802.3ad" mode from the example later seemed to have no effect, likely for similar reasons.
To make matters worse, when I realized my mistake and decided to try switching to a bridge, I kept the bond-lan interface name in place just so I wouldn't have to change a bunch of config files for something I very well might have to change back immediately. This caused Bad Things to happen; systemd-networkd seemed to dislike my interface name, which resulted in the interface being established but not brought up. Changing the name from bond-lan to br1 fixed all my troubles there. I can only guess that I'd somehow fallen afoul of a cache somewhere.
I've got each of these physical ports set to optional in order to allow systemd-networkd to proceed with booting without stopping to wait for a link to be established on each of them, and I have dhcp4 and dhcp6 set to false on them because I don't want them reaching out to whatever's connected for IP addresses. They'll get their IP addresses statically assigned from the bridges defined lower down in the same file.
network:
version: 2
ethernets:
eth0:
dhcp4: no
dhcp6: no
optional: true
eth1:
dhcp4: no
dhcp6: no
optional: true
eth2:
dhcp4: no
dhcp6: no
optional: true
eth3:
dhcp4: no
dhcp6: no
optional: true
eth4:
dhcp4: no
dhcp6: no
optional: true
eth5:
dhcp4: no
dhcp6: no
optional: true
eth6:
dhcp4: no
dhcp6: no
optional: true
eth7:
dhcp4: no
dhcp6: no
optional: true
eth8:
dhcp4: no
dhcp6: no
optional: true
eth9:
dhcp4: no
dhcp6: no
optional: true
bridges:
br0:
interfaces: [eth0, eth6, eth7, eth8, eth9]
addresses: [<ROUTER STATIC CIDR IP>]
br1:
interfaces: [eth1, eth2, eth3, eth4, eth5]
addresses: [192.168.0.1/24]/etc/netplan/10-router.yaml
nameserver <ISP NAMESERVER 1> nameserver <ISP NAMESERVER 2>
/etc/ppp/resolv.conf
# Secrets for authentication using CHAP # client server secret IP addresses "<PPPOE USERNAME>" * "<PPPOE PASSWORD>" *
/etc/ppp/chap-secrets
plugin rp-pppoe.so eth0.<ISP VLAN> unit 1 linkname pppoe user "<PPPOE USERNAME>" noipdefault usepeerdns persist maxfail 0 defaultroute replacedefaultroute hide-password noauth
/etc/ppp/<ISP NAME>
[Unit] Description=PPPoE connection Documentation=man:pon(1) man:pppd(8) man:pppoeconf(8) man:interfaces(5) BindsTo=sys-subsystem-net-devices-eth0.device After=sys-subsystem-net-devices-eth0.device [Service] ExecStartPre=-/sbin/ip link add link eth0 name eth0.<ISP VLAN> type vlan id <ISP VLAN> ExecStartPre=-/sbin/ip link set up dev eth0.<ISP VLAN> ExecStartPre=-/sbin/ip addr add <ROUTER STATIC SUBNET> broadcast <ROUTER STATIC BROADCAST> dev eth0 ExecStartPre=-/sbin/ip route add default via <ROUTER STATIC IP> dev eth0 ExecStart=/usr/sbin/pppd call <ISP NAME> Type=forking SuccessExitStatus=5 PIDFile=/run/ppp-pppoe.pid [Install] WantedBy=sys-subsystem-net-devices-eth0.device
/etc/systemd/system/pppoe.service
{
"Dhcp4": {
"interfaces-config": {
"interfaces": ["br1/192.168.0.1", "br0/<ROUTER STATIC IP>"]
},
"lease-database": {
"type": "memfile",
"persist": true,
"name": "/var/lib/kea/kea-leases4.csv",
"lfc-interval": 3600
},
"renew-timer": 15840,
"rebind-timer": 27720,
"valid-lifetime": 31680,
"option-data": [
{
"name": "domain-search",
"data": "ten64.local"
}
],
"subnet4": [
{
"id": 1,
"subnet": "192.168.0.1/24",
"pools": [ { "pool": "192.168.0.2 - 192.168.0.254" } ],
"option-data": [
{
"name": "routers",
"data": "192.168.0.1"
},
{
"name": "domain-name-servers",
"data": "192.168.0.1"
}
]
},
{
"id": 2,
"subnet": "<STATIC IP SUBNET>",
"pools": [ { "pool": "<STATIC IP #2> - <STATIC IP #N>" } ],
"option-data": [
{
"name": "routers",
"data": "<ROUTER STATIC IP>"
},
{
"name": "broadcast-address",
"data": "<STATIC SUBNET BROADCAST IP>"
},
{
"name": "domain-name-servers",
"data": "<ROUTER STATIC IP>"
}
],
"reservations": [
{ // Home server
"hw-address": "<HOME SERVER NIC MAC ADDRESS>",
"ip-address": "<HOME SERVER STATIC IP>"
}
]
}
]
}
}/etc/kea/kea-dhcp4.conf
With a bunch of comments removed:
Include /etc/ssh/sshd_config.d/*.conf Port 20941 AddressFamily inet #ListenAddress 192.168.0.1 ListenAddress 10.22.20.1 #ListenAddress :: PermitRootLogin no PubkeyAuthentication yes AuthorizedKeysFile .ssh/authorized_keys PasswordAuthentication no KbdInteractiveAuthentication no UsePAM yes X11Forwarding no PrintMotd no # Allow client to pass locale environment variables AcceptEnv LANG LC_* # override default of no subsystems Subsystem sftp /usr/lib/openssh/sftp-server ClientAliveInterval 120
/etc/ssh/sshd_config
For brevity, I've removed all the default values from firewalld.conf below and left in the only value I changed. I prefer dropping packets to rejecting them because I am old school, I guess; also, my network is simple enough that I don't really gain anything from seeing a packet was rejected rather than dropped.
... DefaultZone=drop ...
/etc/firewalld/firewalld.conf
Not that I use the default zone at all, but it's always good to have the defaults you like in place, just in case you miss a specific case.
<?xml version="1.0" encoding="utf-8"?>
<zone target="DROP">
<short>DMZ</short>
<description>Pass-through ports</description>
<source address="<ROUTER STATIC SUBNET>"/>
<service name="mdns"/>
<service name="dns"/>
<protocol value="icmp"/>
<protocol value="ipv6-icmp"/>
<port port="32051" protocol="udp"/>
<port port="63610" protocol="udp"/>
<rule family="ipv4">
<source address="192.168.0.1/24"/>
<masquerade/>
</rule>
<rule family="ipv4">
<source address="10.22.20.0/24"/>
<masquerade/>
</rule>
<interface name="br0"/>
<forward/>
</zone>/etc/firewalld/zones/dmz.xml
<?xml version="1.0" encoding="utf-8"?>
<zone target="DROP">
<short>Internet</short>
<description>A Series of Tubes...</description>
<interface name="eth0"/>
<interface name="eth0.<ISP VLAN>"/>
<interface name="ppp1"/>
<rule family="ipv4">
<destination address="<ROUTER STATIC IP>"/>
<port port="63610" protocol="udp"/>
<accept/>
</rule>
</zone>/etc/firewalld/zones/inet.xml
<?xml version="1.0" encoding="utf-8"?> <zone> <short>LAN</short> <description>Local Area Network</description> <source address="192.168.0.1/24"/> <service name="mdns"/> <service name="dns"/> <service name="dhcp"/> <port port="63610" protocol="udp"/> <protocol value="icmp"/> <protocol value="ipv6-icmp"/> <interface name="br1"/> <forward/> </zone>
/etc/firewalld/zones/lan.xml
<?xml version="1.0" encoding="utf-8"?>
<zone>
<short>LAN</short>
<description>Local Area Network</description>
<source address="192.168.0.1/24"/>
<service name="mdns"/>
<service name="dns"/>
<service name="dhcp"/>
<port port="63610" protocol="udp"/>
<protocol value="icmp"/>
<protocol value="ipv6-icmp"/>
<interface name="br1"/>
<forward/>
</zone>
root@ten64:/etc/firewalld# cat zones/wireguard.xml
<?xml version="1.0" encoding="utf-8"?>
<zone target="DROP">
<short>wg0 zone</short>
<description>Router VPN Wireguard network</description>
<source address="10.22.20.0/24"/>
<service name="dns"/>
<service name="mdns"/>
<protocol value="icmp"/>
<protocol value="ipv6-icmp"/>
<!-- SSH access for specified addresses -->
<rule family="ipv4">
<destination address="10.22.20.1"/>
<port port="20941" protocol="tcp"/>
<source ipset="wg0-admin-ipv4"/>
<accept/>
</rule>
<!-- Interdevice communication for specified addresses -->
<rule family="ipv4">
<destination ipset="wg0-convivial-ipv4"/>
<source ipset="wg0-convivial-ipv4"/>
<accept/>
</rule>
<interface name="wg0"/>
<forward/>
</zone>/etc/firewalld/zones/wireguard.xml
<?xml version="1.0" encoding="utf-8"?> <ipset type="hash:ip"> <short>IPv4s with SSH access</short> <option name="family" value="inet"/> <entry>10.22.20.2</entry> </ipset>
/etc/firewalld/ipsets/wg0-admin-ipv4.xml
<?xml version="1.0" encoding="utf-8"?> <ipset type="hash:ip"> <short>IPv4s allowed to communicate with each other</short> <option name="family" value="inet"/> <entry>10.22.20.2</entry> <entry>10.22.20.3</entry> </ipset>
/etc/firewalld/ipsets/wg0-convivial-ipv4.xml
<?xml version="1.0" encoding="utf-8"?> <ipset type="hash:ip"> <short>IPv4s allowed to masquerade as the router</short> <option name="family" value="inet"/> <entry>10.22.20.2</entry> <entry>10.22.20.3</entry> </ipset>
/etc/firewalld/ipsets/wg0-vpn-ipv4.xml
<?xml version="1.0" encoding="utf-8"?>
<policy target="ACCEPT">
<ingress-zone name="lan"/>
<ingress-zone name="dmz"/>
<ingress-zone name="wireguard"/>
<egress-zone name="inet"/>
<rule>
<tcp-mss-clamp/>
</rule>
<rule family="ipv4">
<source address="192.168.0.0/24"/>
<masquerade/>
</rule>
<rule family="ipv4">
<source ipset="wg0-vpn-ipv4"/>
<masquerade/>
</rule>
</policy>/etc/firewalld/policies/internet-access.xml
<?xml version="1.0" encoding="utf-8"?>
<policy target="DROP">
<egress-zone name="lan"/>
<ingress-zone name="lan"/>
<egress-zone name="dmz"/>
<ingress-zone name="dmz"/>
<egress-zone name="wireguard"/>
<ingress-zone name="wireguard"/>
<!-- Expose Wireguard on the home server, globally -->
<rule family="ipv4">
<destination address="<HOME SERVER STATIC IP>"/>
<port port="32051" protocol="udp"/>
<accept/>
</rule>
<rule family="ipv4">
<source address="192.168.0.0./24"/>
<accept/>
</rule>
<rule family="ipv4">
<source address="10.22.20.0/24"/>
<accept/>
</rule>
</policy>/etc/firewalld/policies/interzone-access.xml
<?xml version="1.0" encoding="utf-8"?> <policy target="DROP"> <ingress-zone name="wireguard"/> <egress-zone name="lan"/> <source ipset="wg0-vpn-ipv4"/> </policy>
/etc/firewalld/policies/wireguard-lan.xml
network:
version: 2
tunnels:
wg0:
mode: wireguard
addresses: [10.22.20.1/24]
port: 63610
key:
private: <ROUTER WIREGUARD PRIVATE KEY>
peers:
- allowed-ips: [10.22.20.2/32]
keys:
public: <LAPTOP WIREGUARD PUBLIC KEY>
- allowed-ips: [10.22.20.3/32]
keys:
public: <PHONE WIREGUARD PUBLIC KEY>/etc/netplan/20-wireguard.yaml
For DNS-based ad blocking at the router I considered using PiHole, but this felt too opaque. Instead, I did something similar to what I'd been doing on the OpenBSD router and set up a script to pull a blocklist from the web, following the instructions at https://github.com/mueller-ma/block-ads-via-dns. I made a few changes to the script there, most notably to switch to RPZ-based blocking, as I'd observed absurdly high memory usage when defining a zone for each blocked domain, as the stock setup does at the moment I'm writing this.
include "/etc/bind/named.conf.options"; include "/etc/bind/named.conf.local"; include "/etc/bind/named.conf.root-hints";
/etc/bind/named.conf, unaltered from its default Debian 13 contents.
acl recursers { localhost; localnets; };
zone "rpz" {
type master;
file "/etc/bind/rpz.db";
};
options {
directory "/var/cache/bind";
response-policy { zone "rpz"; };
forwarders {
<ISP NAMESERVER 1>;
<ISP NAMESERVER 2>;
8.8.8.8;
9.9.9.9;
};
recursion yes;
allow-recursion { recursers; };
//========================================================================
// If BIND logs error messages about the root key being expired,
// you will need to update your keys. See https://www.isc.org/bind-keys
//========================================================================
dnssec-validation auto;
auth-nxdomain no;
listen-on { 192.168.0.1; };
listen-on { <ROUTER STATIC IP>; };
listen-on { 10.22.20.1; };
listen-on-v6 { none; };
check-names master ignore;
check-names slave ignore;
check-names response ignore;
};/etc/bind/named.conf.options
[Unit] Description=Logs system statistics to the systemd journal Wants=block-ads-via-dns.timer [Service] Type=oneshot ExecStart=/root/generate-zonefile.sh [Install] WantedBy=multi-user.target
/etc/systemd/system/block-ads-via-dns.service
[Unit] Description=Refresh the ad blocking DNS zone files Requires=block-ads-via-dns.service [Timer] Unit=block-ads-via-dns.service OnCalendar=*-*-* 07:00:00 [Install] WantedBy=timers.target
/etc/systemd/system/block-ads-via-dns.timer
#!/bin/bash
# Exit immediately if a command exits with a non-zero status
set -e
# display date
date
# Set tempfiles
# All_domains will contain all domains from all lists, but also duplicates and ones which are whitelisted
all_domains=$(mktemp)
# Like above, but no duplicates or whitelisted URLs
all_domains_uniq=$(mktemp)
# We don't write directly to the zonefile. Instead to this temp file and copy it to the right directory afterwards
zonefile=$(mktemp)
zfpreface=$(mktemp)
# Define local black and white lists
# Uncomment if you have no local files
manual_blacklist="/root/blacklist"
manual_whitelist="/root/whitelist"
# StevenBlack GitHub Hosts
# Uncomment ONE line containing the filter you want to apply
# See https://github.com/StevenBlack/hosts for more combinations
#wget -q -O StevenBlack-hosts https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
wget -q -O StevenBlack-hosts https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/fakenews/hosts
#wget -q -O StevenBlack-hosts https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/gambling/hosts
#wget -q -O StevenBlack-hosts https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/porn/hosts
#wget -q -O StevenBlack-hosts https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/social/hosts
#wget -q -O StevenBlack-hosts https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/fakenews-gambling-porn-social/hosts
# Filter out localhost and broadcast
cat StevenBlack-hosts | grep '^0.0.0.0' | egrep -v '127.0.0.1|255.255.255.255|::1' | cut -d " " -f 2 >> $all_domains
# Add local blacklist
if [ -f $manual_blacklist ]
then
cat $manual_blacklist >> $all_domains
fi
# Filter out comments and empty lines
cat $all_domains | egrep -v '^$|#' | sort | uniq > $all_domains_uniq
# Apply local whitelist
if [ -f $manual_whitelist ]
then
for i in $(cat $manual_whitelist)
do
#echo i $i
sed --in-place= "/$i/d" $all_domains_uniq
done
fi
# Populate RPZ database with blocked domains
cat $all_domains_uniq | sed -r 's/(.*)/\1 CNAME ./' > $zonefile # Using RPZ instead of memory-intensive original method.
# Generate RPZ database preface
cat <<EOF > $zfpreface
; Auto-generated from /root/generate-zonefile.sh
; Don't edit manually, you'll lose your changes.
\$TTL 24h
@ IN SOA server.<HOSTNAME>. hostmaster.<HOSTNAME>. (
$(date +%Y%m%d%H) ; Serial
86400 ; Refresh
3600 ; Retry
2419200 ; Expire
10800 ) ; Negative Cache TTL
@ IN NS server.<HOSTNAME>.
EOF
cat $zfpreface $zonefile > /etc/bind/rpz.db
# Remove all tempfiles
rm $all_domains $all_domains_uniq $zonefile $zfpreface StevenBlack-hosts
# Restart bind
systemctl restart named
# For logfile
echo -e 'done\n\n'/root/generate-zonefile.sh
<HOME SERVER NIC MAC ADDRESS>: The MAC address of my home server's NIC, which is plugged directly into the router.<HOME SERVER STATIC IP>: The static IP I've assigned to my home server. E.g., 75.0.0.2<HOSTNAME>: My router's hostname.<ISP NAME>: My ISP's name. Can be whatever you like, really.<ISP NAMESERVER 1>: Provided by my ISP.<ISP NAMESERVER 2>: Provided by my ISP.<ISP VLAN>: Provided by my ISP. The ID of the VLAN over which PPPoE happens.<LAPTOP WIREGUARD PUBLIC KEY>: The public Wireguard key for my laptop.<PHONE WIREGUARD PUBLIC KEY>: The public Wireguard key for my phone.<PPPOE PASSWORD>: Provided by my ISP.<PPPOE USERNAME>: Provided by my ISP.<ROUTER STATIC CIDR IP>: The CIDR rendering of my router's static IP. E.g., 75.0.0.1/29.<ROUTER STATIC IP>: My router's static IP. This is the same as the gateway address in the block of static IP addresses provided by my ISP. E.g., 75.0.0.1.<ROUTER STATIC SUBNET>: The subnet to which my static IPs belong. E.g., 75.0.0.0/29<ROUTER WIREGUARD PRIVATE KEY>: The router's private Wireguard key.<STATIC IP #2>: The first assignable static IP address in my rented block of static IPs, aside fro m the router's own address. E.g., 75.0.0.2<STATIC IP #N>: The last assignable static IP address in my rented block of static IPs.<STATIC IP SUBNET>: The CIDR rendering of my static IP block. E.g., 75.0.0.0/29<STATIC SUBNET BROADCAST IP>: The broadcast address for my static IP block, provided by my ISP.