Setting up Apache SSL with Multiple Certificates on FreeBSD behind a PFSense firewall using NAT

I was setting up SSL on my servers and I've setup SSL before, but only for one domain at a time.  This time, I'd setup SSL to run through a PFSense box to do load balancing.  I have three servers in the web cluster, all running the same software.  So I was planning on setting up the SSL the same way I setup the regular HTTP access, using name based virtual hosting.  I already had five different sites running on the web cluster with no problems.  The PFSense setup was extremely easy.

However, when I added the SSL certificates to the setup, I started getting errors from the client web browsers.  They were complaining that the site certificate mismatched.  And it was true, apache was serving up one SSL cert for all the domains.

I googled around and figured out that you can't use name based virtual hosting for SSL like you can with regular HTTP.  You have to switch to IP based virtual hosting and give each SSL cert its own IP address. 

In PFSense, under Firewall -> Virtual IP, I created a new external IP address for example2.com domain, since I already had an IP created for example1.com domain.

Then I went to Services -> LoadBalancer to create the pools.  A pool is the list of internal servers that will handle the web requests.  I had already created a pool for example1.com  They were being served by 10.10.1.5, 10.10.1.6, and 10.10.1.7.  I labeled it "example1 secure".

Setting Up Alias IPs on FreeBSD

However, I couldn't use the same set of IPs for this new pool, even though I wanted to use the same servers.  So, I created a set of alias IP addresses on the FreeBSD servers.  When you use name based virtual servers, you specify the external name even if its inside NAT.  We can't do that when we are behind a NAT firewall. We have to use IP addresses on the FreeBSD web servers that are inside the NAT range.  These IP addresses will be used to differentiate the SSL traffic to apache on the web cluster.  All the IP addresses in this tutorial are fictitious and you should adjust accordingly.

On the 10.10.1.5 box, I added this line to /etc/rc.conf

ifconfig_bge0_alias0="10.10.1.15 netmask 255.255.255.255"

This tells FreeBSD to add a second IP address to the existing network card.  The netmask of 255.255.255.255 is important, because traffic doesn't really go out this address.  Its just another name for the main address already assigned.  The main address will have a netmask that fits on the network and because I didn't want to reboot, I used this at the command line to set it up immediately:

ifconfig bge0 inet 10.10.1.15 netmask 255.255.255.255 alias

Then I repeated it for the other boxes. On server 10.10.1.6, I added:

ifconfig_bge0_alias0="10.10.1.16 netmask 255.255.255.255"

On server 10.10.1.7, I added:

ifconfig_bge0_alias0="10.10.1.17 netmask 255.255.255.255"

Now each server had 2 IP addresses inside the firewall.  We will send the SSL traffic for example1.com to 10.10.1.5 and SSL traffic for example2.com to 10.10.1.15.  (Repeat for each server in the cluster...)

Setting Up PFSense Load Balancing

So now I went back to Services -> LoadBalancer to create the new pool.  I created a new pool for example2.com for load balancing servers on port 443.

Next I setup the Virtual Server.  This is how the external IP gets mapped to the pool of internal servers.  You don't even need to setup NAT.  Its all handled internally by PFSense.

I put in the external IP address of example2.com and configured it for port 443.  By selecting "Example2 Secure", the pool that we just setup, it routes the traffic to the internal servers.

Setting Up Apache

This part of the setup will need to be repeated on each server in the cluster, but substitute out the right IP addresses for each server.

I modified the httpd.conf file to include these next sections:

<virtualhost 10.10.1.5:443>
SSLEngine On
ServerName example1.com
ServerAlias www.example1.com
ServerAdmin webmaster@example1.com
DocumentRoot /usr/local/data/example1/webroot
ErrorLog /usr/local/data/logs/error.log
TransferLog /usr/local/data/logs/access.log
SSLCertificateFile /usr/local/etc/apache22/ssl/example1
SSLCertificateKeyFile /usr/local/etc/apache22/ssl/example1
SSLCACertificateFile /usr/local/etc/apache22/ssl/intermediate.crt
</virtualhost>

<virtualhost 10.10.1.15:443>
SSLEngine On
ServerName example2.com
ServerAlias www.example2.com
ServerAdmin webmaster@example2.com
DocumentRoot /usr/local/data/example2/webroot
ErrorLog /usr/local/data/logs/error.log
TransferLog /usr/local/data/logs/access.log
SSLCertificateFile /usr/local/etc/apache22/ssl/example2
SSLCertificateKeyFile /usr/local/etc/apache22/ssl/example2
SSLCACertificateFile /usr/local/etc/apache22/ssl/intermediate.crt
</virtualhost>

The first one sets up 10.10.1.5 to handle SSL for example1.com and the 2nd sets up 10.10.1.15 to handle example2.com.  Now apache can tell the two domains apart by the IP address the traffic is getting sent to.

Then restart apache:

apachectl restart

Don't forget to add a firewall rule to allow HTTP traffic on port 443 to these new servers.  All I had to do was add the new IP addresses to the web server alias.  I really like that feature of PF.