diff --git a/content/drafts/.htaccess b/content/drafts/.htaccess new file mode 100644 index 0000000..e21f9fe --- /dev/null +++ b/content/drafts/.htaccess @@ -0,0 +1,7 @@ +--- +extends: plain.j2 +--- + +AddHandler send-as-is .asis +Options +MultiViews +MultiviewsMatch Handlers Filters diff --git a/content/drafts/isolated-ip-management.html.asis b/content/drafts/isolated-ip-management.html.asis new file mode 100644 index 0000000..4ba2662 --- /dev/null +++ b/content/drafts/isolated-ip-management.html.asis @@ -0,0 +1,4 @@ +--- +extends: redirect.j2 +location: https://blog.funkthat.com/posts/isolated-ip-management.html +--- diff --git a/content/posts/isolated-ip-management.html b/content/posts/isolated-ip-management.html new file mode 100644 index 0000000..1393963 --- /dev/null +++ b/content/posts/isolated-ip-management.html @@ -0,0 +1,165 @@ +--- +title: Isolated IP Management +description: > + For routers, a management interface should always work, even if the + firewall is broken, or the IP address conflicts with other interfaces. + Isolating the interface to a vnet jail and using unix domain sockets fixes + this problem. +posted: !!timestamp '2023-08-17' +created: !!timestamp '2023-08-17' +time: 12:00 PM +tags: + - FreeBSD + - jails +--- + +The latest iteration of my home firewall has a spare interface. Many +switches these days have a dedicated management interface, and I wanted +something similar for my firewall. I want an interface that I can plug in, +get an IP via DHCP, and I can ssh into and it'll "just work" even if +there's a misconfiguration, or an IP conflict. + +On FreeBSD, +[vnet jails](https://wiki.freebsd.org/Jails#VNET-based_networking_for_Jails) +are a way to isolate an interface such that it won't interfere (or receive +interference) from other interfaces. Some may have just used epair and +assigned IPs, but this could cause conflict, while using a unix domain +socket keeps things isolated, and FreeBSD ships w/ everything needed to +make this work. A package like `socat` isn't needed. + +Configuring the host was straight forward, make a directory for the socket: +``` +mkdir /var/mgmt +``` + +And then add a line to `/etc/inetd.conf` to listen to incoming connections +on the socket and launch `sshd`: +``` +/var/mgmt/mgmt.ssh.sock stream unix nowait root /usr/sbin/sshd sshd -i +``` + +The `-i` option to `sshd` tells it to run in "`inetd`" mode, which means that +the stdin and stdout of the process is the socket to use for communication. + +The next harder part was getting a jail configured that would accept +incoming connections and forward them to the unix domain socket. + +First the jail configuration, which goes in `/etc/jail.conf` or similar +location: +``` +mgmt { + host.hostname = mgmt; # Hostname + + vnet ="new"; + vnet.interface="mgmt0"; + + path = "/usr/jails/mgmt/root"; # Path to the jail + mount.fstab="/usr/jails/mgmt/fstab"; # mount spec + + mount.devfs; # Mount devfs inside the jail + devfs_ruleset = "101"; + + exec.start = "/bin/sh /etc/rc"; # Start command + exec.stop = "/bin/sh /etc/rc.shutdown"; # Stop command +} +``` + +There isn't anything special in this. It's pretty standard jail +configuration, the differences are the vnet configuration, and the +`devfs_ruleset`. + +The `devfs_ruleset` is necessary in order to expose the bpf interface +used by dhclient. This required the following lines in `/etc/devfs.rules`: +``` +[mydevfsrules_jail=100] +add include $devfsrules_hide_all +add include $devfsrules_unhide_basic +add include $devfsrules_unhide_login + +[devfsrules_jail_dhcp=101] +add include $mydevfsrules_jail +add path 'bpf*' unhide +``` + +Note that after adding the above lines, you need to run: +``` +service devfs start +``` +to load the rules (per +[devfs(8)](https://man.freebsd.org/cgi/man.cgi?query=devfs&apropos=0&sektion=8&manpath=FreeBSD+13.2-RELEASE+and+Ports&arch=default&format=html) +man page). + +Note: I learned my mistake not to number my blocks immediately after +the standard defaults (from `/etc/defaults/devfs.rules`). I had done that +befure but there's now a conflict, so I skip ahead a bit to get a unique range. + +Now I needed to make a number of directories for the jail: +``` +mkdir -p /usr/jails/mgmt/root +mkdir -p /usr/jails/mgmt/etc +mkdir -p /usr/jails/mgmt/var/mgmt +mkdir -p /usr/jails/mgmt/tmp +``` + +I needed to setup the `fstab` for the jail: +``` +# Device Mountpoint FStype Options Dump Pass# +/ /usr/jail/mgmt/root nullfs ro 0 0 +/usr/jail/mgmt/etc /usr/jail/mgmt/root/etc nullfs rw 0 0 +/usr/jail/mgmt/var /usr/jail/mgmt/root/var nullfs rw 0 0 +/var/mgmt /usr/jail/mgmt/root/var/mgmt nullfs ro 0 0 +/usr/jail/mgmt/tmp /usr/jail/mgmt/root/tmp nullfs rw 0 0 +``` + +This is a little bit more tricky, It first `nullfs` mounts the root system. +I'm using ZFS boot environments, so this is a pretty clean FreeBSD install +without much host specific data. It then mounts some jail specific directories +for `etc`, `tmp` and `var` and finally mounts the shared directory w/ the +unix domain socket to the host system. Also note that a couple of the mounts +are read-only to prevent the jail from modifying the system. + +The `etc` directory was populated from the system `etc` via: +``` +tar -cf - -C /etc . | tar -xf - -C /usr/jails/mgmt/etc +``` + +Then the jail was configured, first `/usr/jails/mgmt/etc/rc.conf`: +``` +hostname="mgmt.funkthat.com" + +sendmail_enable="NONE" +sendmail_submit_enable="NO" +sendmail_outbound_enable="NO" +sendmail_msp_queue_enable="NO" + +# Management port +ifconfig_mgmt0="DHCP" + +# necessary as devd can't be run in a jail +synchronous_dhclient="YES" + +inetd_enable="YES" # Run the network daemon dispatcher (YES/NO). +``` + +The key part of this configuration that took me a while to figure out was +the `synchronous_dhclient` line. It used to be that `netif` would start +dhclient, but in order to better handle USB ethernet devices and other +removable interfaces, it was moved to `devd`. The only problem is that +`devd` hasn't been jail'ified, and you can't run it to get things like +link notifications that would normally launch dhclient. Setting this to +yes, makes sure it gets launched when the jail starts. + +And then the following line was added to `/usr/jails/mgmt/etc/inetd.conf`: +``` +ssh stream tcp nowait root /usr/bin/nc nc -N -U /var/mgmt/mgmt.ssh.sock +``` + +This is the part that will forward incoming connections to the ssh port +on to the unix domain socket. + +Now that everything is configured, a simple `jail -c mgmt` will get +the jail running and accepting connections. + +This was testing and deployed on a FreeBSD 14-CURRENT build as of August +8th, 2023, or more specifically, from `main-n264621-09c20a29328`, but it +should work on all currently supported FreeBSD releases. diff --git a/content/posts/meta.yaml b/content/posts/meta.yaml new file mode 100644 index 0000000..675e1bd --- /dev/null +++ b/content/posts/meta.yaml @@ -0,0 +1,3 @@ +extends: blog.j2 +default_block: post +listable: true diff --git a/layout/plain.j2 b/layout/plain.j2 new file mode 100644 index 0000000..b24bca7 --- /dev/null +++ b/layout/plain.j2 @@ -0,0 +1,2 @@ +{% block post %} +{% endblock post %} diff --git a/layout/redirect.j2 b/layout/redirect.j2 new file mode 100644 index 0000000..53da3cc --- /dev/null +++ b/layout/redirect.j2 @@ -0,0 +1,15 @@ +Status: 301 Moved Permanently +Location: {{ resource.meta.location }} +Content-type: text/html + + + +New location + + +

This page has been relocated to: +here. +

+ + +