vzfirewall with ISPConfig3

jesse's picture

This article shows how to setup vzfirewall on an OpenVZ hardware node with ISPConfig3.

I'm testing out ISPConfig3 as a replacement for DTC, which has gone inactive and it seems time to move on. I had setup a hardware node on Debian Wheezy following the howtoforge article, then created containers for web, mail, mysql and dns. I had enabled the firewall for the hardware node, which configured the integrated bastille firewall that ISPConfig3 ships with, then switched ISPConfig from bastille to ufw as I like ufw better.

ISPConfig Firewall problems

ISPConfig Firewall: too limited

But the firewall rules ISPConfig created are just too limited/open. Also, by default the ufw firewall rules are completely incompatible with running containers.

This ISPConfig instance is only intended to manage my hadware nodes, not for customer access, so I opened the requisite ports in the Firewall configuration. The resulting ufw rules create are:

Chain ufw-user-input (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:22
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:80
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:443
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:3306
  115  6900 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:8080
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:8081

The main problem here is there is no ability to restrict what source addresses can access these ports - I sure don't want mysqld open to the world, and in this case I want to restrict access to all the others as well.

Then the ISPConfig/ufw firewall problem using containers is that the container traffic all passes the FORWARD chain in iptables, which ends up being blocked. You can edit /etc/default/ufw and set DEFAULT_FORWARD_POLICY="ACCEPT" to leave FORWARD wide open, and that would actually work fine if you setup a firewall inside each container.

Solution: vzfirewall

Short of adding better firewall support right to ISPConfig, the best option I can think of is vzfirewall, a firewall script built specifically for use with OpenVZ. vzfirewall will let me setup the firewall for the hardware node properly, hard code the correct rules for my infrastructure containers, and then leave any other containers (eg. for customer VPS) open so another firewall can be used inside them as needed.

So lets remove ufw and setup vzfirewall.

Remove UFW

Login to ISPConfig and under System > Firewall, delete the firewall record for this host. Then ssh to the server, and as root run:

service fail2ban stop
ufw disable
iptables -F INPUT
iptables -F FORWARD
iptables -F OUTPUT
iptables -L | grep 'Chain ufw' | cut -d' ' -f2 | xargs -n1 iptables -F
iptables -L | grep 'Chain ufw' | cut -d' ' -f2 | xargs -n1 iptables -X
apt-get purge ufw -y
service fail2ban start

Install vzfirewall

I'm going to use vzfirewall from my own repo here, as there's a patch for veth networking that's not been merged upstream yet.

apt-get install iptables-persistent
 
cd /usr/sbin
wget https://github.com/jnorell/vzfirewall/raw/master/vzfirewall
chmod +x vzfirewall
 
cd /etc/vz
wget https://github.com/jnorell/vzfirewall/raw/master/vzfirewall.conf
 
cd /etc/vz/conf
(test -f vps.premount && echo "vps.premount exists, manual integration required") || ( \
    wget https://github.com/jnorell/vzfirewall/raw/master/vps.premount; \
    chmod +x vps.premount )

That's it, and even that last file isn't necessary, as we're about to overwrite it.

vzfirewall/ISPConfig integration

As ISPConfig doesn't support arbitrary config in the virtual server config, nor the NETFILTER or #FIREWALL items explicitly, we'll create some vzctl actions scripts to add that in for us. We'll make them smart enough to check for an existing config and not re-add it, so if/when ISPConfig does support that in the future, we're still safe.

After following the above to install vzfirewall, you will already have a /etc/vz/conf/vps.premount script, we'll just modify that. First lets make a directory for default and per-container configuration addons, then edit or replace /etc/vz/conf/vps.premount to include the new "addon" section:

mkdir /etc/vz/conf/addon
 
cat > /etc/vz/conf/vps.premount <<'EOF'
#!/bin/bash
#
# vps.premount
#
 
#  Sanity check the environment
[ -z "${VEID}" ] && exit 1
[ -z "${VE_CONFFILE}" ] && exit 1
 
#  Source VPS configuration files in the same order as vzctl
[ -f /etc/vz/vz.conf ] || exit 1
[ -f ${VE_CONFFILE} ] || exit 1
. /etc/vz/vz.conf
. ${VE_CONFFILE}
 
if [ -d /etc/vz/conf/addon ]; then
  cd /etc/vz/conf/addon
 
  # NETFILTER= in .netfilter
  if ! grep -q ^NETFILTER= ${VE_CONFFILE}; then
    if [ -f ${VEID}.netfilter ]; then
      cat ${VEID}.netfilter >> ${VE_CONFFILE}
    elif [ -f default.netfilter ]; then
      cat default.netfilter >> ${VE_CONFFILE}
    fi
  fi
 
  # #FIREWALL= in .firewall
  if ! grep -q ^\#FIREWALL= ${VE_CONFFILE}; then
    if [ -f ${VEID}.firewall ]; then
      cat ${VEID}.firewall >> ${VE_CONFFILE}
    elif [ -f default.firewall ]; then
      cat default.firewall >> ${VE_CONFFILE}
    fi
  fi
 
fi
 
if type -p vzfirewall > /dev/null; then
  #  Test vzfirewall config for errors before running
  if vzfirewall -t > /dev/null 2>&1; then
    vzfirewall -a > /dev/null 2>&1
  fi
fi
 
exit 0
EOF
 
chmod +x /etc/vz/conf/vps.premount 

We'll create "addon" configuration files below.

NETFILTER

We want NETFILTER="stateful" added to each container config file is so a stateful firewall can be used inside each container:

echo 'NETFILTER="stateful"' > /etc/vz/conf/addon/default.netfilter

You could set individual containers to "disabled" if you had a reason to do so (performance/memory?), or even "full" if a container needed to perform NAT.

Configure vzfirewall

An understanding of vzfirewall configuration syntax is necessary, see the README.txt for an example. The commands below could be cut/pasted into a shell with no cleanup; notably, the leading # in the FIREWALL= setting should stay there, as vzfirewall config is saved as comments in the individual container .conf files.

vzfirewall for hardware node

Our hardware node only needs access to a few ports from a private administrative subnet, which will include the other hardware nodes acting as ISPConfig slaves. As a failsafe, vzfirewall will open port 22 by default (specify FAILSAFE_ADDRS in /etc/vz/vzfirewall.conf to restrict that), so we don't need that here:

cat >> /etc/vz/conf/0.conf <<'EOF'
 
#FIREWALL="
#   # ISPConfig access from admin subnets
#   [80,443,8080,8081]
#   192.168.100.0/24
#   192.168.200.0/24
#   my.admin.hostname
#
#   # mysql access from other hardware nodes (ISPConfig slaves)
#   [3306]
#   192.168.100.0/26
#"
EOF

You can run vzfirewall -a to implement those rules now.

vzfirewall for service containers

I have 5 containers setup for ISPConfig services, for web, mail, mysql and 2 dns, with container id's 101-105. We'll put the firewall configuration into an "addon" file so it gets re-added if ISPConfig ever overwrites it (see above).

web container

A customer instance of ISPConfig will run on the web server port 8080, and we'll allow ports 40110-40210 for ftp, so container 101 config is:

cat > /etc/vz/conf/addon/101.firewall <<'EOF'
 
#FIREWALL="
#   [20,21,22,80,443,8080,40110:40210]
#   *
#
#   # mysql access from other service containers (ISPConfig slaves)
#   [3306]
#   192.168.100.128/25
#"
EOF

mail container

Email ports are open to all, ssh from our admin subnets, and mysql from the master ISPConfig server. Container 102 config is thus:

cat > /etc/vz/conf/addon/102.firewall <<'EOF'
 
#FIREWALL="
#   [25,110,143,465,587,993,995]
#   *
#   # SSH access from admin subnets
#   [22]
#   192.168.100.0/24
#   192.168.200.0/24
#   my.admin.hostname
#
#   # mysql access from ISPConfig master
#   [3306]
#   192.168.100.130
#   cp.admin.hostname
#"
EOF

mysql container

Here mysqld is open to all the customer infrastructure addresses, including the master ISPConfig server, and ssh from our admin subnets. Container 103 config:

cat > /etc/vz/conf/addon/103.firewall <<'EOF'
 
#FIREWALL="
#   # SSH access from admin subnets
#   [22]
#   192.168.100.0/24
#   192.168.200.0/24
#   my.admin.hostname
#
#   # mysql access from all customer infrastructure and ISPConfig master
#   [3306]
#   192.168.100.128/25
#   cp.admin.hostname
#"
EOF

dns containers

DNS requests are open from the world, mysql from the master ISPConfig server, and ssh from our admin subnets. Container 104 and 105 config is:

cat > /etc/vz/conf/addon/104.firewall <<'EOF'
 
#FIREWALL="
#   # DNS
#   [udp:53]
#   *
#   [53]
#   *
#
#   # SSH access from admin subnets
#   [22]
#   192.168.100.0/24
#   192.168.200.0/24
#   my.admin.hostname
#
#   # mysql access from ISPConfig master
#   [3306]
#   192.168.100.130
#   cp.admin.hostname
#"
EOF
 
cp /etc/vz/conf/addon/104.firewall /etc/vz/conf/addon/105.firewall

vzfirewall left open for customer containers

We'll configure the default for other (ie. customer) containers to be wide open. That will let them use their own firewall setup inside the container, be it ufw or something else.

cat > /etc/vz/conf/addon/default.firewall <<'EOF'
 
#FIREWALL="
#   # wide open here, so run a firewall inside the container
#   *
#"
EOF

restart containers

Everything is now setup to be configured as containers start, all that's left is to restart them all:

vzlist -1 | xargs -n 1 vzctl restart

And that's it - each container conf file should have a NETFILTER and #FIREWALL section, and vzfirewall -a will be run by the vps.premount to setup all the firewall rules.

Closing thoughts

ISPConfig is looking promising in initial testing, though both its firewall and OpenVZ virtual server support are a little too basic for my taste/needs. Some of the roadmap items will definitely help. The setup here will help in the interim. I'll probably leverage something similar to the "addon" solution above in order to utilize multiple ip addresses, until that issue is resolved.

If you have problems or a better approach, feel free to use the comments below.

Tags: 

Add new comment

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
By submitting this form, you accept the Mollom privacy policy.