Bastille jail manager Notes

Background

So I have previously been running a handfull jails using the iocage jail manager. Ken Moore from IX Systems stated that they stopped collaborating on the project and the few other dev’s apparently didn’t have the time and energy to push forward. I don’t blame them, they have done a great job and I really love iocage so big kudos to them.

I started having some issues with the release py39-iocage pkg, so I switched to the py39-iocage-devel under the latest pkg branch. This seemed to work for a while, while hoping that someone would pick up the project.
I have now stumbled upon a bug which stopped me from upgrading my jails, when running the iocage fetch command, it stopped at: Fetching 2 metadata files… failed.

I tried the best I could troubleshooting, but with no luck.

Meanwhile the BastilleBSD project seems to have gained traction.
So I decided to look into how to migrate the jails fra iocage to bastille. And how to get things running in a way that was acceptable for the future use of jails.
A bit of research proved that iocage had kind of gone down a path moving away from the original way of managing jails and instead having their own way of managing the jails. Using iocage conf files instead of jail conf files.
Bastille is still close to the roots of jails, and everything in bastille is written as shell scripts instead of Python.
Other jail management systems that can be used is:

Pot – Github page
cbsd – Github page

Bastille have okay documentations, but the project is still new and a lot of development are going on making some docs obsolete and some functions haven’t got any docs. So this is my try to get people started with Bastille going from iocage.

Getting started – Migrating

Migrating from iocage is almost straight forward.

iocage stop <jail>
iocage export <jail>

This gives you your exported jail with a compressed datafile containing your whole zfs dataset, and a checksum file which is used to verify your compressed file when importing into Bastille.

Install Bastille on your host machine:

pkg install bastille
sysrc bastille_enable=YES

#Setup your /etc/rc.conf with correct network
sysrc cloned_interfaces+=lo1
sysrc ifconfig_lo1_name="bastille0"
service netif cloneup

Bastille rely on pf as firewall, but I use VNET in all my jails, so we set a very permissive firewall rules.

nano /etc/pf.conf

# Copy the following rules:

ext_if="vtnet0"

set block-policy return
scrub in on $ext_if all fragment reassemble
set skip on lo

table <jails> persist
nat on $ext_if from <jails> to any -> ($ext_if:0)
rdr-anchor "rdr/*"


#block in all
pass out quick keep state
#antispoof for $ext_if inet #Comment this out when using VNET.
pass in proto { tcp udp } to port { 22 53 80 123 443 8080 8443 }
pass in inet proto tcp from any to any port ssh flags S/SA keep state
pass in inet proto tcp from any to any port bootps flags S/SA keep state
pass in inet proto tcp from any to any port {8080,9100,9124} flags S/SA keep state

Be sure to the variable ext_if=”vtnet0″ to your network interface.
I am using a FreeBSD host that is hosted on a Proxmox System. But it could also be em0 or igb0 etc. etc.
I am not that into pf settings, so if anyone has suggestions to other settings in this, please write to me.

Enable the rules:

nano /etc/rc.conf

## copy inside the rc.conf file.

pf_enable="YES"
pf_rules="/etc/pf.conf"
pfflags=""
pflog_enable="YES"
pflog_logfile="/var/log/pflog"
pflog_flags=""

gateway_enable="YES"

Start pf by service pf restart

I love ZFS, and I think it should be the standard filesystem on every system out there.

sysrc -f /usr/local/etc/bastille/bastille.conf bastille_zfs_enable=YES
sysrc -f /usr/local/etc/bastille/bastille.conf bastille_zfs_zpool=ZPOOL_NAME

Replace ZPOOL_NAME with the zpool you want to use for bastille. In my case I use the zroot

VNET requires a custom defs ruleset, so you need to create the file /etc/devfs.rules

[bastille_vnet=13]
add path 'bpf*' unhide

You need to bootstrap bastille with the FreeBSD version that your jails are on, you can later upgrade those after importing.

bastille bootstrap 13.1-RELEASE

You are now ready to import your iocage jails into bastille.

Copy the 2 files ../iocage/images/<jail>.zip and ../iocage/images/<jail>.sha256 into /usr/local/bastille/backups

cp ../iocage/images/* /usr/local/bastille/backups/

Import the jail:

root@hostname: # bastille import <jail>.zip
Validating file: <jail>.zip
File validation successful!
Importing '<jail>' from foreign compressed .zip archive.
Archive: <jail>.zip
 extracting: <jail>
 extracting: <jail>_root
Receiving ZFS data stream...
Generating jail.conf...
Container '<jail>' imported successfully.

This takes a while depending of the size of your jail.

Sometimes the jail.conf file do not get all the informations inside. So verify that the following lines are correct:

  vnet;
  vnet.interface = e0b_bastille0;
  exec.prestart += "jib addm bastille0 vtnet0";

Now its time to make some manual adjustments to the jail to have VNET enabled as expected.

bastille start <jail>
bastille console <jail>

# configure the jail /etc/rc.conf file with correct ip
# Lookup the network interface
ifconfig

# Should be some like e0b_bastille0
# Edit /etc/rc.conf

ifconfig_e0b_bastille0_name="vnet0"
ifconfig_vnet0="inet IP_ADDRESS netmask 255.255.255.0"
defaultrouter="IP_TO_GATEWAY"

# Exit out of the jail to the host
exit

Restart the jail with the new settings.

bastille restart <jail>

# Verify the ip with
bastille list -a

If your jail is old, you can upgrade it with the following command:

bastille upgrade <jail> 13.1-RELEASE

Follow the instructions.

You should now have successfully migrated your iocage jails into bastille.

Edit 1:
I have later found that bastille do not handle MAC addresses when using VNET in jails.
So you need to edit the jail.conf file in each jail folder by adding 2 extra exec.prestart lines:

  vnet;
  vnet.interface = e0b_bastille1;
  exec.prestart += "jib addm bastille1 vtnet0";
  exec.prestart += "ifconfig e0a_bastille1 description \"vnet host interface for Bastille jail mariadb\"";
  exec.prestart += "ifconfig e0a_bastille1 192.168.11.14/24";
  exec.prestart += "ifconfig e0a_bastille1 ether 58:9c:fc:bc:9c:f3";
  exec.poststop += "jib destroy bastille1";

The first extra prestart line is to have the jail IP on the interface on the host. Which is nice, when quickly running a ifconfig. The second is to assign a MAC address other than the host to the jail interface. My system ran extremely slow when not setting individual MAC addresses to each jail interface.