pfsense

Integrating Remote Servers Into My Local Network

1 Comment

This post may contain affiliate links. Please see the disclaimer for more information.

I’ve been using Linode for many years to host what I consider to be my most “production grade” self-hosted services, namely this blog and my mail server. My initial Linode server was built in 2011 on CentOS 6. This is approaching end of life so, I’ve been starting to build its replacement. Since originally building this server my home network has grown up and now provides a myriad of services. When starting out to build the new server, I thought it would be nice to be able to make use of these more easily from my remote servers. So I’ve begun some work to integrate the two networks more closely.

Integration Points

There are a few integration points I’m targeting here, some of which I’ve done already and others are still to be done:

  • Get everything onto the same network, or at least on different subnets of my main network so I can control traffic between networks via my pfSense firewall. This gives me the major benefit of being able to access selected services on my local network from the cloud without having to make that service externally accessible. In order to do this securely you want to make sure the connection is encrypted – i.e. you want a VPN. I’m using OpenVPN.
  • Use ZFS snapshots for backing up the data on the remote systems. I’d previously been using plain old rsync for copying the data down locally where it gets rolled into my main backups using restic. There is nothing wrong with this approach, but using ZFS snapshots gives more flexibility for restoring back to a certain point without having to extract the whole backup.
  • Integrate into my check_mk setup for monitoring. I’m already doing this and deploying the agent via Ansible/CI. However, now the agent connection will go via the VPN tunnel (it’s SSH anyway, so this doesn’t make a huge difference).
  • Deploy the configuration to everything with Ansible and Gitlab CI – I’m still working on this!
  • Build a centralised logging server and send all my logs to it. This will be a big win, but sits squarely in the to-do column. However, it will benefit from the presence of the VPN connection, since the syslog protocol isn’t really suitable for running over the big-bad Internet.

Setting Up OpenVPN

I’m setting this up with the server being my local pfSense firewall and the clients being the remote cloud machines. This is somewhat the reverse to what you’d expect, since the remote machines have static IPs. My local IP is dynamic, but DuckDNS does a great job of not making this a problem.

The server setup is simplified somewhat due to using pfSense with the OpenVPN Client Export package. I’m not going to run through the full server setup here – the official documentation does a much better job. One thing worth noting is that I set this up as the second OpenVPN server running on my pfSense box. This is important as I want these clients to be on a different IP range, so that I can firewall them well. The first VPN server runs my remote access VPN which has unrestricted access, just as if I were present on my LAN.

In order to create the second server, I just had to select a different UDP port and set the IP range I wanted in the wizard. It should also be noted that the VPN configuration is set up not to route any traffic through it by default. This prevents all the traffic from the remote server trying to go via my local network.

On the client side, I’m using the standard OpenVPN package from the Ubuntu repositories:

$ sudo apt install openvpn

After that you can extract the configuration zip file from the server and test with OpenVPN in your terminal:

$ unzip <your_config>.zip
$ cd <your_config>
$ sudo openvpn --config <your_config>.ovpn

After a few seconds you should see the client connect and should be able to ping the VPN address of the remote server from your local network.

Always On VPN Connection

To make this configuration persistent we first move the files into /etc/openvpn/client, renaming the config file to give it the .conf extension:

$ sudo mv <your_config>.key /etc/openvpn/client.key
$ sudo mv <your_config>.p12 /etc/openvpn/client.p12
$ sudo mv <your_config>.ovpn /etc/openvpn/client.conf

You’ll want to update the pkcs12 and tls-auth lines to point to the new .p12 and .key files. I used full paths here just to makes sure it would work later. I also added a route to my local network in the client config:

route 10.0.0.0 255.255.0.0

You should then be able to activate the OpenVPN client service via systemctl:

$ sudo systemctl start openvpn-client@client.service
$ sudo systemctl enable openvpn-client@client.service

If you check your system logs, you should see the connection come up again. It’ll now persist across reboots and should also reconnect if the connection goes down for any reason. So far it’s been 100% stable for me.

At this point I added a DNS entry on my pfSense box to allow me to access the remote machine via it’s hostname from my local network. This isn’t required, but it’s quite nice to have. The entry points to the VPN address of the machine, so all traffic will go via the tunnel.

Firewall Configuration

Since these servers have publicly available services running on them, I don’t want them to have unrestricted access to my local network. Therefore, I’m blocking all incoming traffic from the new VPN’s IP range in pfSense. I’ll then add specific exceptions for the services I want them to access. This is pretty much how you would set up a standard DMZ.

remote server integrate
The firewall rules for the OpenVPN interface, note the SSH rule to allow traffic for our ZFS snapshot sync later

To do this I added an alias for the IP range in question and then added a block rule on the OpenVPN firewall tab in pfSense. This is slightly different to the way my DMZ is set up, since I don’t want to block all traffic on the OpenVPN interface, just traffic from that specific IP range (to allow my remote access VPN to continue working!).

You’ll probably also want to configure the remote server to accept traffic from the VPN so that you can access any services on the server from your local network. Do this with whatever Linux firewall tool you usually use (I use ufw).

Storing Data on ZFS

And now for something completely different….

As discussed before, I was previously backing up the data on these servers with rsync. However, I was missing the snapshotting I get on my local systems. These local systems mount their data directories via NFS to my main home server, which then takes care of the snapshot duties. I didn’t want to use NFS over the VPN connection for performance reasons, so I opted for local snapshots and ZFS replication.

In order to mount a ZFS pool on our cloud VM we need a device to store our data on. I could add some block storage to my Linodes (and I may in future), but I can also use a loopback file in ZFS (and not have to pay for extra space). To do this I just created a 15G blank file and created the zpool on top of that:

$ sudo mkdir /zpool
$ sudo dd if=/dev/zero of=/zpool/storage bs=1G count=15
$ sudo apt install zfsutils-linux
$ sudo zpool -m /storage storage /zpool/storage

I can then go about creating my datasets (one for the mail storage and one for docker volumes):

sudo zfs create storage/mail
sudo zfs create storage/docker-data

Automating ZFS Snapshots

To automate my snapshots I’m using Sanoid. To install it (and it’s companion tool Syncoid) I did the following:

$ sudo apt install pv lzop mbuffer libconfig-inifiles-perl libconfig-inifiles-perl git
$ git clone https://github.com/jimsalterjrs/sanoid
$ sudo mv sanoid /opt/
$ sudo chown -R root:root /opt/sanoid
$ sudo ln /opt/sanoid/sanoid /usr/sbin/
$ sudo ln /opt/sanoid/syncoid /usr/sbin/

Basically all we do here is install a few dependencies and then download Sanoid and install it in /opt. I then hard link the sanoid and syncoid executables into /usr/sbin so that they are on the path. We then need to copy over the default configuration:

$ sudo mkdir /etc/sanoid
$ sudo cp /opt/sanoid/sanoid.conf /etc/sanoid/sanoid.conf
$ sudo cp /opt/sanoid/sanoid.defaults.conf /etc/sanoid/sanoid.defaults.conf

I then edited the sanoid.conf file for my setup. My full configuration is shown below:

[storage/mail]
        use_template=production

[storage/docker-data]
        use_template=production
        recursive=yes

#############################
# templates below this line #
#############################

[template_production]
        frequently = 0
        hourly = 36
        daily = 30
        monthly = 12
        yearly = 2
        autosnap = yes
        autoprune = yes

This is pretty self explanatory. Right now I’m keeping loads of snapshots, I’ll pare this down later if I start to run out of disk space. The storage/docker-data dataset has recursive snapshots enabled because I will most likely make each Docker volume its own dataset.

This is all capped off with a cron job in /etc/cron.d/zfs-snapshots:

*  *    * * *   root    TZ=UTC /usr/local/bin/log-output '/usr/sbin/sanoid --cron'

Since my rant a couple of weeks ago, I’ve been trying to assemble some better practices around cron jobs. The log-output script is one of these, from this excellent article.

Syncing the Snapshots Locally

The final part of the puzzle is using Sanoid’s companion tool Syncoid to sync these down to my local machine. This seems difficult to do in a secure way, due to the permissions that zfs receive needs. I tried to use delegated permissions, but it looks like the mount permission doesn’t work on Linux.

The best I could come up with was to add a new unprivileged user and allow it to only run the zfs command with sudo by adding the following via visudo:

syncoid ALL=(ALL) NOPASSWD:/sbin/zfs

I also set up an SSH key on the remote machine and added it to the syncoid user on my home server. Usually I would restrict the commands that could be run via this key for added security, but it looks like Syncoid does quite a bit so I wasn’t sure how to go about this (if any one has any idea let me know).

With that in place we can test our synchronisation:

$ sudo syncoid -r storage/mail syncoid@<MY HOME SERVER>:storage/backup/mail
$ syncoid -r storage/docker-data syncoid@<MY HOME SERVER>:storage/docker/`hostname`

For this to work you should make sure that the parent datasets are created on the receiving server, but not the destination datasets themselves, Syncoid likes to create them for you.

I then wrote a quick script to automate this which I dropped in /root/replicator.sh:

#!/bin/bash

USER=syncoid
HOST=<MY HOME SERVER>

HOSTNAME=$(hostname)

/usr/sbin/syncoid -r storage/mail $USER@$HOST:storage/backup/mail 2>&1
/usr/sbin/syncoid -r storage/docker-data $USER@$HOST:storage/docker/$HOSTNAME 2>&1

Then another cron job in /etc/cron.d/zfs-snapshots finishes the job:

56 *    * * *   root    /usr/local/bin/log-output '/root/replicator.sh'

Conclusion

Phew! There was quite a bit there. Thanks for sticking with me if you made it this far!

With this setup I’ve come a pretty long way towards my goal of better integrating my remote servers. So far I’ve only deployed this to a single server (which will become the new mailserver). There are a couple of others to go, so the next step will be to automate as much as possible of this via Ansible roles.

I hope you’ve enjoyed this journey with me. I’m interested to hear how others are integrating remote and local networks together. Let me know if you have anything to add via the feedback channels.

If you liked this post and want to see more, please consider subscribing to the mailing list (below) or the RSS feed. You can also follow me on Twitter. If you want to show your appreciation, feel free to buy me a coffee.

By subscribing, you agree with our privacy policy and our terms of service.

pfsense proxmox open vswitch

Virtualised pfSense on Proxmox with Open vSwitch

9 Comments

This post may contain affiliate links. Please see the disclaimer for more information.

In my recent post about my networking setup I mentioned that my firewall is a virtualised pfSense system running on a Proxmox host. In the comments to that post I was also asked if I was making use of Open vSwitch. Since the answer is that I use Open vSwitch in my pfSense/Proxmox setup, I thought I’d write up my setup for those that are interested.

I’ve actually been meaning to write this up for a long time. I’ve had this setup running since shortly after we moved into this house. On the one hand this means that the setup is pretty battle tested. All of the inter-VLAN and Internet bound traffic on my network runs through this and it’s been running pretty flawlessly for nearly two years.

On the other hand, given the length of time that has elapsed since I set this up and the writing of this post it means that this will be more like archeological exploration than documentation! I’m unlikely to remember every detail or the issues I encountered along the way. As such this post will pretty much document the state of the setup as I can extract it from the running system! Basically, you should only use this post as a rough guide and go away and do your own research. I’ll apologise for this incompleteness in advance. If you try this please let me know of anything I’ve missed and I’ll update the post with extra details.

How’s this going to work?

The basic premise of this whole thing is a Proxmox host with two physical NICs. One of these is the LAN port on which the host will have it’s internal IP. The second is the WAN port, which is assigned directly to the pfSense VM. In my case this is complicated by networking setup required by our Fibre connections here in NZ. These require a connection to the Fibre ONT on VLAN10 over which a PPPoE session to the ISP is established.

Since the WAN interface is directly assigned to the VM, this is all handled internally to pfSense. This means that that the host machine is not exposed to the external network. [OK, for the purists among you, this isn’t strictly true. The host will be exposed at lower levels of the network stack to allow it to forward packets through to the VM. However, since it doesn’t have an IP address on that interface it won’t be accessible from the Internet. I’m sure someone out there will tell me why this is all kinds of horrible.]

On the LAN side we create an Open vSwitch switch and add the LAN interface as a VLAN trunk on it. Another (virtual) trunk interface goes into the pfSense VM and becomes it’s LAN interface. This is analogous to just having another physical switch between the host and the VM. The purpose of this extra complexity is that it allows us to connect other VMs on the host into the vSwitch. These can be in on multiple different VLANs if required.

Hopefully the diagram below makes this somewhat clearer:

pfsense proxmox open vswitch
Sometimes a picture really is worth a thousand words

The Proxmox Host

The Proxmox host itself is a Dual Ethernet Haswell based mini-computer from AliExpress. I’ve been really happy with this as a platfrom aside from the fact that I would have spec’d it with more than 4GB of RAM if I’d been intending to run Proxmox initially. I also added an extra 120GB SSD drive on the internal SATA port for VM storage.

I started out with this host running pfSense natively, which also worked fine. One thing I did find is that when I switched over to Proxmox (Linux based) from pfSense (FreeBSD based) it ran much cooler. I guess that’s just down the the Linux kernel’s better hardware support.

This host is still running Proxmox 5.4 since I haven’t had time to upgrade it to 6.0 yet. This system is pretty much as close to “production” as it gets for me, since the Internet is used all the time!

Proxmox Network Setup

Proxmox enumerates the two NICs as ens1 (LAN) and enp1s0 (WAN). With the WAN port, I created a simple Linux Bridge vmbr1 to allow it to be added to the pfSense VM.

On the LAN side, I created an “OVS Bridge” port and added an “OVS IntPort” named admin which will be the primary interface to the host machine. As such, this interface is assigned a static IP and is assigned to the VLAN that we want the host to be on.

pfsense proxmox open vswitch
The network setup in Proxmox

I have to give kudos here to the Proxmox developers. They’ve made the Open vSwitch setup here pretty much trivial! For what I would consider advanced functionality it’s just as easy as configuring any other network.

A note should also be given here as to what’s going to happen when you configure this. By design Proxmox doesn’t apply any networking changes until you reboot. This is pretty useful to prevent yourself getting locked out. If you are connected directly on the LAN interface (with a static IP) you should make sure that everything is correct before rebooting. After the reboot, reconfigure your local interface to the VLAN you chose in the setup and a static IP. You should then be able to access the Proxmox web interface again.

Setting Up pfSense

The pfSense installation was fairly standard. The only change I ended up making was to change the default CPU type to enable AES-NI instructions. This took a little bit of experimentation and looking up the capabilities of various processors, but I finally settled on the “Westmere” processor.

pfsense proxmox open vswitch
I selected a “Westmere” processor in order to make AES-NI instructions available to the VM

After setting this architecture in the VM settings, rebooting pfSense shows both the correct CPU architecture and that AES-NI is available. It seems that this is probably less important than it was when I set up the system, since Netgate have now decided that AES-NI will not be required for pfSense 2.5.0.

pfsense proxmox open vswitch
AES-NI successfully enabled

One other thing is that you should disable hardware checksum offloading to work with the virtio drivers, as per the official documentation. Before you do this the network will be very sluggish.

Once the pfSense installation was complete I restored from a backup of my previous setup. This made the task of setting up my interfaces significantly easier. However, I’ll go through the networking aspects anyway for those who may be setting up a new system.

pfSense Networking

Luckily for us the pfSense tool to assign interfaces allows us to also set up the VLANs. This is useful to set up a minimal configuration to get you access to the web interface. Basically you want to set up the VLAN for your main LAN segment. Then you can set up the pfSense LAN interface on this VLAN with a static IP. If you’re using a fibre connection similar to mine you can skip the WAN setup for now. Once the “Assign Interfaces” wizard is complete you should have access to the Web Configurator.

The next step was to setup my WAN connection. I first added a VLAN with tag 10 on the vtnet0 device which is the device that corresponds to the physical WAN bridge as enumerated by pfSense. I added a corresponding interface for this and then added a PPPoE interface using the details provided by my ISP. This is then assigned to the WAN interface via the “Interface Assignments” page.

In terms of setting up the local networks, you can pretty much set up whatever VLANs you would like at this point. Take a look at my previous post for inspiration.

Conclusion

As stated earlier, I’ve found this setup to be very stable in production and it’s even made my hardware run cooler. Having my firewall virtualised has also had several other benefits for me. Firstly, I can backup and snapshot the firewall VM at will. I no longer need to worry about an update or bad configuration hosing my firewall. I just snapshot before doing anything major and roll back if anything goes wrong.

The second major benefit is that I can run extra VMs and containers on the host machine, which I couldn’t when it was a dedicated firewall. I’ve used this to implement my small DMZ for Internet facing services. This has the added benefit that DMZ traffic only transits the vSwitch internal to the host and doesn’t have to be shuttled back and forth over the physical network infrastructure. This is much faster, since the virtualised interfaces should (in theory) be 10GBps. However, this is somewhat irrelevant when the upstream Fibre connection is only 100Mbps.

As always, I’m keen to receive feedback and constructive criticism of this setup. Please feel free to get in contact via the feedback channels.

If you liked this post and want to see more, please consider subscribing to the mailing list (below) or the RSS feed. You can also follow me on Twitter. If you want to show your appreciation, feel free to buy me a coffee.

By subscribing, you agree with our privacy policy and our terms of service.

smarthome network

Building a Smarthome Network with Open Source Software

7 Comments

This post may contain affiliate links. Please see the disclaimer for more information.

I’ve recently been doing some network restructuring and clean up in order to better separate devices on my network and to remove a bit of the cruft that builds up over time. In the process, I realised I haven’t written about how my smarthome network is structured. My network setup is somewhat different to lots of other setups I’ve seen documented. This is because I’m mostly using Open Source software to drive my firewall and wireless access points.

This article will introduce the technologies I’m using on my network and give you an overview of the structure. It’s not intended to be a full how-to, but I’ll try to include links which will guide you though any parts I don’t cover in detail. Let’s get into it…

What’s special about a Smarthome Network?

The typical smarthome today includes a myriad of devices, probably from a variety of different manufacturers. Obviously, there will be your smarthome devices – smart bulbs, switches, hubs/gateways, vacuums, IP cameras, etc. However, there are all the standard devices too – smartphones, tablets, laptops, desktops, etc. Additionally, it’s likely that there will be some devices specifically used for media consumption.

We can see that there are obviously different classes of devices in the lists above. These differ both in function and in the level of trust that you give to them. It’s this attribute of trust that separates a smarthome network from a standard home network for me.

Consider this, are there any devices on your home network that you maybe don’t trust all that much. What about that light bulb over there? When did it last get a security update? Do you really want that sitting on the same network as the devices you trust with your personal communications, important documents and web searches?

Using existing technologies, we can partition our network in such a way as to separate our trusted and untrusted devices from one another. We don’t need to stop there! We can separate devices that require Internet access from those that don’t or along any other lines that we want. The technologies in question are well trusted and have been around for years: VLANs and inter-network firewalling.

Open Source Components

I’ve been operating a partitioned network setup for some time, pretty much since I got seriously into Home Automation technologies. Recently I’ve seen renewed interest in this is the community. The recent series from Rob at The Hook Up on YouTube was particularly good (part 1, part 2, part 3). A large part of the community appears to be using the Unifi line of products from Ubiquity. I’m not questioning the quality or performance of these products – I’ve never tried them. However, I prefer to use Open Source alternatives where possible.

smarthome network
You shall not pass!

The primary components of my smarthome network are running Open Source software. The first of these is my firewall, running pfSense. The second of these are my wireless access points. For these I use a couple of consumer grade TP-Link wireless routers, running OpenWRT. Since these are running solely as access points and Ethernet switches they don’t suffer performance issues from the consumer hardware. I also don’t put excessive strain on my wireless, preferring to use Ethernet wherever possible.

smarthome network
The road goes ever on…

Unfortunately, there doesn’t appear to be an Open Source OS available for readily available Ethernet switches. As an alternative I’m using a couple of TP-Link switches (the TL-SG1024DE and TL-SG105E). These are great switches for the price. Crucially they come with the VLAN capability that we need for our partitioned smarthome network.

Network Partitioning With VLANs

For those unfamiliar with VLANs, they feel like magic when you discover them! A VLAN is a virtual network, which runs over the top of your physical network cables. Several of these networks can be run over the same connection. This means we can run several networks on the same physical hardware. VLANs work by tagging each Ethernet frame with a network identifier so that the receiving hardware knows which network to send it on to. This usually requires hardware support in your switching hardware, although software support is available in most operating systems.

I subdivide my network into several VLANs, based around both trust and functionality:

  • The main LAN network, this houses the trusted client devices (laptops, smartphones, etc). This network has access to most of the others for maintenance purposes.
  • The IOT network, which houses smart devices which absolutely require Internet access. At the moment this is just my Smart TV, Neato Botvac and a Chromecast Ultra. The Neato is the only device that uses the associated WLAN. This network is firewalled from all the others but is allowed to access the Internet.
  • The NOT (Network of Things – name stolen from The Hook Up video series, linked above). This houses smart devices which are locally controlled, such as my ESPHome devices. Some of the devices on this network (such as my Yeelight bulbs, Milight Gateway and Broadlink RM Mini) want to get out to the Internet but are blocked by the firewall. For ease of use I put the VM hosting my Home Automation services on this network and make an exception for it in the firewall. I’m hoping to change this eventually. This network is blocked both from the other local networks and the Internet, aside from a few exceptions.
  • The Media network, this houses all the devices and servers which stream media around my house. This includes both my Kodi systems, an older HDHomerun and the Emby, tvheadend and Mopidy/Snapcast servers. For now the RPi driving my outdoor speakers is living in the NOT network. This is because it’s on wifi and I didn’t want to create another AP for it. Eventually that will be migrated over, once I run a cable into the roof for it. Having the media devices in a separate VLAN should allow me to do some traffic shaping in the future to prioritise their traffic (if needed, I don’t have any problems right now). This network isn’t blocked from any of the others and has Internet access.
  • The DMZ, this hosts any services which are available from outside my network. It’s blocked from all local networks, but has Internet access.
  • The Guest network, which is tied directly to a specific WLAN only for guest devices. This allows me to provide Internet access to guests without giving them access to anything else. Blocked from all the local networks, but obviously has Internet access.
  • The Infrastructure network, this one is new and I’m still migrating devices over to it. The idea is that it will house all the network infrastructure devices, including switches, access points and the physical host servers. Everything in here will have a static IP address. Right now the firewall is open. I will probably lock it down so that admin can only be performed from trusted devices.
  • The Servers network, this one is also new and so far empty. Eventually it will contain all the internal server VMs (i.e. non-DMZ, media or HA). The idea is that I can use firewall rules to control access to these from other parts of the network. Most definitely a work in progress.
  • The Work network, which houses my company workstation and any other devices used for work, since I work from home. This is blocked from the other networks, except for a few ports. It obviously has Internet access.

Phew… that’s quite a …few(!).

VLANs With pfSense

VLANs with pfSense are fairly easy to configure. It’s basically a two step process:

  1. Create the VLAN in Interfaces->Assignments->VLANs
  2. Add a net interface which uses the VLAN in Interfaces->Assignments
smarthome network
The pfSense VLANs page
smarthome network
The pfSense interfaces page, this maps VLANs to interfaces used in firewall rules

The pfSense documentation gives a better overview of this.

Once your VLANs and interfaces are available you should be able to configure the firewall rules to control traffic between them. You’re probably going to want to block access to the local networks from your secure VLANs and potentially allow Internet access. If you want to also deny Internet access just deny access from that network to all destinations.

smarthome network
The firewall rules for my IOT network

OpenWRT: VLANs and Multiple APs

The main reason for using OpenWRT on my wireless APs is to unlock capabilities of the underlying hardware that aren’t available in the stock firmware. Specifically, this will allow you to create multiple wireless access points and assign them to different VLANs.

smarthome network
Multiple WLAN Access Points in OpenWRT

The basic process here is to create a bridge interface. This can be used to group your VLAN and the wireless network together. Adding the VLANs themselves is pretty trivial. There is even a nice GUI editor which shows each port and the corresponding VLANs.

smarthome network
The OpenWRT VLAN assignments interface is the best I’ve found so far

The main gotcha of this setup is to make sure that the dnsmasq service is disabled in System->Startup to prevent it interfering with the DNS and DHCP from the firewall. You can also check the “Ignore Interface” DHCP setting in the interface config for each interface.

You can also delete the default created WAN interface to allow you to use the fifth switch port on the router as another port on the network. This should also disable NATing between the ports which you also don’t need with this setup. Basically, once you are finished tweaking all the settings the device will only act as an access point (for multiple wireless networks) and managed switch.

Special Considerations

There aren’t too many issues that you should run into with this setup, once you get all the pieces into place. There were a few minor things which tripped me up however (two of these are specific to the switch I’m using):

  • My TL-SG1024DE switch is a little funny about what VLAN it’s management interface will be available on. It seems like it’s VLAN 1 or nothing. Additionally it adds VLAN 1 as untagged to every port. I eventually just made VLAN 1 into my Infrastructure network to work around this.
  • The TL-SG1024DE switch also requires you to set the PVID setting on each port, even if the port will never receive any untagged frames. I just set it to whatever the primary VLAN of the port is.
  • I held off moving the Chromecast into the IOT network for a long time, since I was worried about the discovery not working across subnets. As it turns out, I shouldn’t have worried. All you have to do is install and enable the Avahi package in pfSense and you’re good to go. This video helped me out, though the settings seem to have been simplified since then (see the screenshot below for my settings).
smarthome network
Avahi settings in pfSense

Core Network and Topology

I’m not going to cover how I setup the main switch on my network, since the purpose of this article is to focus on the Open Source components. Your configuration will also vary depending on what switch you have.

I will say a few words on the physical topology of my smarthome network. My firewall and main switch are located in my ‘rack’. The firewall machine has two network interfaces. One is used as the WAN port and connected to my fibre ONT. The other is the main VLAN trunk to the switch. In my case my pfSense install is actually installed as a virtual machine hosted by Proxmox which runs on the host machine. This is somewhat irrelevant to this discussion and probably deserves a post all of its own.

From the main switch, connections go out to the various rooms in the house. In three of these I have additional switches where I need the extra ports. These are the two OpenWRT devices and the TL-SG105E switch. This layout also gives nice wireless coverage around the house. The uplinks to each of these are configured as VLAN trunks for whichever networks are required at the other end.

Conclusions

Hopefully this post has shown you that it’s possible to create a fully featured and secure smarthome network using Open Source components. I’ve probably glossed over tons of the details of my setup in the process of writing this article. If I put them all in, it would probably be three times as long! If you have any questions, please ask them in the feedback channels. I’m also not a professional network engineer, so feel free to provide improvements and constructive criticism!

My network is a constant work in progress. However, once this latest round of tidying up is complete I think I’ll be in good shape for quite a while. Hopefully this network will happily support all the future projects I have planned for my smarthome!

If you liked this post and want to see more, please consider subscribing to the mailing list (below) or the RSS feed. You can also follow me on Twitter. If you want to show your appreciation, feel free to buy me a coffee.

By subscribing, you agree with our privacy policy and our terms of service.