Building a Smarthome Network with Open Source Software

September 10, 2019Rob Connolly 7 Comments

This post may contain affiliate links. Please see the disclaimer for more information.

I’ve recently been doing some network restructuring and clean up in order to better separate devices on my network and to remove a bit of the cruft that builds up over time. In the process, I realised I haven’t written about how my smarthome network is structured. My network setup is somewhat different to lots of other setups I’ve seen documented. This is because I’m mostly using Open Source software to drive my firewall and wireless access points.

This article will introduce the technologies I’m using on my network and give you an overview of the structure. It’s not intended to be a full how-to, but I’ll try to include links which will guide you though any parts I don’t cover in detail. Let’s get into it…

What’s special about a Smarthome Network?

The typical smarthome today includes a myriad of devices, probably from a variety of different manufacturers. Obviously, there will be your smarthome devices – smart bulbs, switches, hubs/gateways, vacuums, IP cameras, etc. However, there are all the standard devices too – smartphones, tablets, laptops, desktops, etc. Additionally, it’s likely that there will be some devices specifically used for media consumption.

We can see that there are obviously different classes of devices in the lists above. These differ both in function and in the level of trust that you give to them. It’s this attribute of trust that separates a smarthome network from a standard home network for me.

Consider this, are there any devices on your home network that you maybe don’t trust all that much. What about that light bulb over there? When did it last get a security update? Do you really want that sitting on the same network as the devices you trust with your personal communications, important documents and web searches?

Using existing technologies, we can partition our network in such a way as to separate our trusted and untrusted devices from one another. We don’t need to stop there! We can separate devices that require Internet access from those that don’t or along any other lines that we want. The technologies in question are well trusted and have been around for years: VLANs and inter-network firewalling.

Open Source Components

I’ve been operating a partitioned network setup for some time, pretty much since I got seriously into Home Automation technologies. Recently I’ve seen renewed interest in this is the community. The recent series from Rob at The Hook Up on YouTube was particularly good (part 1, part 2, part 3). A large part of the community appears to be using the Unifi line of products from Ubiquity. I’m not questioning the quality or performance of these products – I’ve never tried them. However, I prefer to use Open Source alternatives where possible.

The primary components of my smarthome network are running Open Source software. The first of these is my firewall, running pfSense. The second of these are my wireless access points. For these I use a couple of consumer grade TP-Link wireless routers, running OpenWRT. Since these are running solely as access points and Ethernet switches they don’t suffer performance issues from the consumer hardware. I also don’t put excessive strain on my wireless, preferring to use Ethernet wherever possible.

Unfortunately, there doesn’t appear to be an Open Source OS available for readily available Ethernet switches. As an alternative I’m using a couple of TP-Link switches (the TL-SG1024DE and TL-SG105E). These are great switches for the price. Crucially they come with the VLAN capability that we need for our partitioned smarthome network.

Network Partitioning With VLANs

For those unfamiliar with VLANs, they feel like magic when you discover them! A VLAN is a virtual network, which runs over the top of your physical network cables. Several of these networks can be run over the same connection. This means we can run several networks on the same physical hardware. VLANs work by tagging each Ethernet frame with a network identifier so that the receiving hardware knows which network to send it on to. This usually requires hardware support in your switching hardware, although software support is available in most operating systems.

I subdivide my network into several VLANs, based around both trust and functionality:

The main LAN network, this houses the trusted client devices (laptops, smartphones, etc). This network has access to most of the others for maintenance purposes.
The IOT network, which houses smart devices which absolutely require Internet access. At the moment this is just my Smart TV, Neato Botvac and a Chromecast Ultra. The Neato is the only device that uses the associated WLAN. This network is firewalled from all the others but is allowed to access the Internet.
The NOT (Network of Things – name stolen from The Hook Up video series, linked above). This houses smart devices which are locally controlled, such as my ESPHome devices. Some of the devices on this network (such as my Yeelight bulbs, Milight Gateway and Broadlink RM Mini) want to get out to the Internet but are blocked by the firewall. For ease of use I put the VM hosting my Home Automation services on this network and make an exception for it in the firewall. I’m hoping to change this eventually. This network is blocked both from the other local networks and the Internet, aside from a few exceptions.
The Media network, this houses all the devices and servers which stream media around my house. This includes both my Kodi systems, an older HDHomerun and the Emby, tvheadend and Mopidy/Snapcast servers. For now the RPi driving my outdoor speakers is living in the NOT network. This is because it’s on wifi and I didn’t want to create another AP for it. Eventually that will be migrated over, once I run a cable into the roof for it. Having the media devices in a separate VLAN should allow me to do some traffic shaping in the future to prioritise their traffic (if needed, I don’t have any problems right now). This network isn’t blocked from any of the others and has Internet access.
The DMZ, this hosts any services which are available from outside my network. It’s blocked from all local networks, but has Internet access.
The Guest network, which is tied directly to a specific WLAN only for guest devices. This allows me to provide Internet access to guests without giving them access to anything else. Blocked from all the local networks, but obviously has Internet access.
The Infrastructure network, this one is new and I’m still migrating devices over to it. The idea is that it will house all the network infrastructure devices, including switches, access points and the physical host servers. Everything in here will have a static IP address. Right now the firewall is open. I will probably lock it down so that admin can only be performed from trusted devices.
The Servers network, this one is also new and so far empty. Eventually it will contain all the internal server VMs (i.e. non-DMZ, media or HA). The idea is that I can use firewall rules to control access to these from other parts of the network. Most definitely a work in progress.
The Work network, which houses my company workstation and any other devices used for work, since I work from home. This is blocked from the other networks, except for a few ports. It obviously has Internet access.

Phew… that’s quite a …few(!).

VLANs With pfSense

VLANs with pfSense are fairly easy to configure. It’s basically a two step process:

Create the VLAN in Interfaces->Assignments->VLANs
Add a net interface which uses the VLAN in Interfaces->Assignments

The pfSense documentation gives a better overview of this.

Once your VLANs and interfaces are available you should be able to configure the firewall rules to control traffic between them. You’re probably going to want to block access to the local networks from your secure VLANs and potentially allow Internet access. If you want to also deny Internet access just deny access from that network to all destinations.

OpenWRT: VLANs and Multiple APs

The main reason for using OpenWRT on my wireless APs is to unlock capabilities of the underlying hardware that aren’t available in the stock firmware. Specifically, this will allow you to create multiple wireless access points and assign them to different VLANs.

The basic process here is to create a bridge interface. This can be used to group your VLAN and the wireless network together. Adding the VLANs themselves is pretty trivial. There is even a nice GUI editor which shows each port and the corresponding VLANs.

The main gotcha of this setup is to make sure that the dnsmasq service is disabled in System->Startup to prevent it interfering with the DNS and DHCP from the firewall. You can also check the “Ignore Interface” DHCP setting in the interface config for each interface.

You can also delete the default created WAN interface to allow you to use the fifth switch port on the router as another port on the network. This should also disable NATing between the ports which you also don’t need with this setup. Basically, once you are finished tweaking all the settings the device will only act as an access point (for multiple wireless networks) and managed switch.

Special Considerations

There aren’t too many issues that you should run into with this setup, once you get all the pieces into place. There were a few minor things which tripped me up however (two of these are specific to the switch I’m using):

My TL-SG1024DE switch is a little funny about what VLAN it’s management interface will be available on. It seems like it’s VLAN 1 or nothing. Additionally it adds VLAN 1 as untagged to every port. I eventually just made VLAN 1 into my Infrastructure network to work around this.
The TL-SG1024DE switch also requires you to set the PVID setting on each port, even if the port will never receive any untagged frames. I just set it to whatever the primary VLAN of the port is.
I held off moving the Chromecast into the IOT network for a long time, since I was worried about the discovery not working across subnets. As it turns out, I shouldn’t have worried. All you have to do is install and enable the Avahi package in pfSense and you’re good to go. This video helped me out, though the settings seem to have been simplified since then (see the screenshot below for my settings).

Core Network and Topology

I’m not going to cover how I setup the main switch on my network, since the purpose of this article is to focus on the Open Source components. Your configuration will also vary depending on what switch you have.

I will say a few words on the physical topology of my smarthome network. My firewall and main switch are located in my ‘rack’. The firewall machine has two network interfaces. One is used as the WAN port and connected to my fibre ONT. The other is the main VLAN trunk to the switch. In my case my pfSense install is actually installed as a virtual machine hosted by Proxmox which runs on the host machine. This is somewhat irrelevant to this discussion and probably deserves a post all of its own.

From the main switch, connections go out to the various rooms in the house. In three of these I have additional switches where I need the extra ports. These are the two OpenWRT devices and the TL-SG105E switch. This layout also gives nice wireless coverage around the house. The uplinks to each of these are configured as VLAN trunks for whichever networks are required at the other end.

Conclusions

Hopefully this post has shown you that it’s possible to create a fully featured and secure smarthome network using Open Source components. I’ve probably glossed over tons of the details of my setup in the process of writing this article. If I put them all in, it would probably be three times as long! If you have any questions, please ask them in the feedback channels. I’m also not a professional network engineer, so feel free to provide improvements and constructive criticism!

My network is a constant work in progress. However, once this latest round of tidying up is complete I think I’ll be in good shape for quite a while. Hopefully this network will happily support all the future projects I have planned for my smarthome!

If you liked this post and want to see more, please consider subscribing to the mailing list (below) or the RSS feed. You can also follow me on Twitter. If you want to show your appreciation, feel free to buy me a coffee.

Self-Hosted GPS Tracking with Traccar and Home Assistant

September 3, 2019Rob Connolly 1 Comment

This post may contain affiliate links. Please see the disclaimer for more information.

One of the outstanding issues I’ve had with my Home Automation system in recent months has been to fix my presence detection. I was using a combination of Owntracks and SNMP to query my Pfsense firewall for devices on the network.

This worked well until my Owntracks setup broke due to internal network changes. This was due to moving separating my HA/MQTT server from the reverse proxy. In turn, this meant that the MQTT server wasn’t able to renew it’s Let’s Encrypt certificate.

The solution to this would have been to use DNS validation to get the certificate issued. However, my DNS provider (Namecheap) don’t allow API access unless you have a large enough account with them.

Eventually I will migrate my DNS to a supported provider and get internal TLS working. However, it’s quite a bit of work to migrate over. Also, DNS is pretty critical to the operation of, well, everything – so I want to take my time and get it right. In the meantime I was looking into other options for GPS based presence detection.

GPS Based Presence Options

Aside from using Owntracks in MQTT mode, it also supports HTTP mode (which is now actually the recommended mode). This is directly supported in HASS. I hadn’t switched over to it because I wanted to try out the Owntracks recorder for logging position data over time. Sending data to both HASS and the recorder would be difficult via HTTP. However, it’s exactly the use case that MQTT is made for!

While I was dithering around not doing very much about this problem, another alternative cropped up: Traccar. Traccar is a self-hosted GPS tracking system which supports a multitude of different devices and has mobile apps available for both major OSes. The main plus point for me is that it is a stand-alone server which I could host in my DMZ. There is also a Home Assistant integration for Traccar.

Installation

It seems to be rather badly documented, but there is an official Docker image for Traccar on Docker Hub. When reading that documentation I initially thought I had to build the image myself. Don’t! Just use the official one (unless you have a good reason not to).

I followed along the instructions in the Readme, until it came to the final docker run command, where I wanted to put it into docker-compose. Here’s what I came up with:

---
version: '3'

services:
  traccar:
    image: traccar/traccar:latest
    restart: always
    ports:
      - "8082:8082"
      - "5000-5150:5000-5150"
      - "5000-5150:5000-5150/udp"
    volumes:
      - "/mnt/docker-data/traccar/logs:/opt/traccar/logs:rw"
      - "/mnt/docker-data/traccar/data:/opt/traccar/data:rw"
      - "/mnt/docker-data/traccar/traccar.xml:/opt/traccar/conf/traccar.xml:rw"

This is pretty much the command in the docs translated over, except for the second to last line. This mounts the data volume needed for persisting the default H2 database files to the host. I’m not sure why this isn’t mentioned in the docs. Perhaps they expect that you will use Mysql for the database, but I didn’t want to do that for my initial test setup.

After running the docker-compose up -d command I had Traccar working on port 8082 of my server and was able to log in as the default user (admin and password admin!). The first thing I did after logging in was change that password and disable user registration, before exposing my instance via the reverse proxy.

I found that the server memory usage was reasonably high, which seems to be due to the memory options passed to the Java VM in the dockerfile. There doesn’t seem to be any way to change this except for building a custom image.

Reverse Proxy Setup

I’m intending to migrate the reverse proxy on my home network to Traefik at some point after my previous success with it. However, for now it’s still running good old Nginx. I couldn’t find an example Nginx config for Traccar, so I copied my HASS one and modified it to suit:

server {
    # Update this line to be your domain
    server_name traccar.example.com;

    # Ensure these lines point to your SSL certificate and key
    ssl_certificate /etc/letsencrypt/live/traccar.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/traccar.example.com/privkey.pem;

    # Ensure this line points to your dhparams file
    ssl_dhparam /etc/nginx/ssl/dhparams.pem;

    # These shouldn't need to be changed
    listen 443 ;
    add_header Strict-Transport-Security "max-age=31536000; includeSubdomains";
    ssl on;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4";
    ssl_prefer_server_ciphers on;
    ssl_session_cache shared:SSL:10m;

    proxy_buffering off;

    location / {
        # Update the address of your backend server here
        proxy_pass http://backend-server:8082;
        proxy_set_header Host $host;
        proxy_redirect http:// https://;
        proxy_http_version 1.1;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
    }
}

Adding Devices

Once your server is up and running, adding devices is easy. Just click the plus icon next to devices and give your device a name. For the ID field, the Android app will generate a six-digit identifier, which you can copy over. This ID is the only security token used to update the position, so you may like to use something more secure. I recommend generating a pseudo-random string with pwgen and using that.

Set up on the app side is pretty trivial too. Just enter the full URL to the server e.g. https://traccar.example.com and make sure the device identifier matches the one you entered on the server. Toggling the service status will then start sending location updates. The Android app seems to put an annoying persistent notification in the notification pull down. For some reason this doesn’t go into the ongoing section (which would make it less annoying). I just hid the notifications from the Traccar app via the relevant Android setting and the app still sends location updates.

As an aside, you can easily send location updates from the command line with curl, for example I made the screenshot above by artificially positioning a device at New Plymouth’s Wind Wand:

curl -X POST "https://<insert-hostname>/?id=<insert-device-id>&lat=-39.056056&lon=174.071736&hdrop=0&altitude=4&speed=0"

Make sure to update the hostname and device ID to match your setup and update the other fields to reflect the position you want to log.

Integrating Traccar with Home Assistant

The integration with HASS also proved to be relatively easy and works well. There were a couple of things which tripped me up, which I’ll come to shortly. First we need an entry for Traccar in our Home Assistant config:

device_tracker:
  - platform: traccar
    host: !secret traccar_host
    port: 443
    username: !secret traccar_user
    password: !secret traccar_password
    ssl: true

The port and ssl variables are required for the above setup with the reverse proxy, aside from that it’s reasonably obvious.

After adding that to my config file, I expected my devices just to show up in HASS. They didn’t. This turned out to be due to two problems. The first of these was that I had created a user in Traccar specifically for Home Assistant (which you can give read-only permissions, nicely). However, this user didn’t have access to my devices. The solution is to grant access via the ‘Devices’ panel of the user management panel (the icon looks like two little picture frames).

The second issue tripped me up for a bit longer. Even after I set the permissions correctly, I still couldn’t see my devices in the HASS UI. It turns out they were being added to the known_devices.yaml on the server, but not being enabled by default. It wasn’t until I logged into the server and checked this file that I noticed this. This is more problematic if you are editing your config locally and deploying it via git since the change obviously won’t be made to your local copy.

In the end I added the entry in my local copy and deployed it to the server:

robs_phone:                                                                                                                                                                                     
  hide_if_away: false                                                                                                                                                                           
  icon:                                                                                                                                                                                         
  mac:                                                                                                                                                                      
  name: Rob                                                                                                                                                                                     
  picture:                                                                                                                                                                                      
  track: true

Once that was done for each device, the Traccar device_tracker entities appeared in Home Assistant just fine.

Battery Usage Issues

Now comes the big potential problem: the power usage of the Traccar Android app. The first day I had this installed my phone was down to 15% by 9-10pm. However, the results have been less worrying over the last couple of days. I’m not sure why it was so bad for the first day, although there were a few points that were different from my subsequent usage:

I was using the version from F-Droid. After noticing the battery issue I switched to the version from the play store. I don’t know if that version uses some proprietary Google Services API, or if they are identical. However, it is a possible reason for the difference.
I had changed a couple of the default settings – I lengthened the frequency setting to 15 minutes and changed the distance setting to 50 meters. I’m not sure what effect/interactions these have since I can’t find any documentation about them. I’ve since gone back to the default settings.
I moved around quite a bit that first day (some driving around town and a long walk) with less moving in the subsequent two days. If this is really the reason for the battery usage then that would be disappointing. Obviously not moving is not a solution!

This is all based on three days of usage, so I’m going to continue monitoring the situation and potentially testing other variations in settings. I’ve asked about the battery usage on the Traccar forum, but have so far had no answer. One worrying issue is that the Android battery usage panel shows the app as keeping the device awake for long periods.

There is another option to the Traccar app, since OwnTracks can also be used. However, at this point you may as well just use the HASS integration directly, unless you want the extra features of the Traccar server.

Conclusion

Traccar seems like a pretty solid piece of software, though due to the power issue I’m not completely sold on it yet. The location updates are certainly more frequent than OwnTracks was (at least over MQTT). The server side UI is nice and responsive and there are quite a few capabilities that I haven’t explored yet. For these reasons I’m going to stick with it for now and continue testing.

Next Steps

Obviously I need to solve the battery issues in order to go much further. If the usage proves acceptable after further testing I’m also going to see if I can dial down the memory usage of the server by editing the Dockerfile.

I also have quite a bit of work to do on the HASS presence detection front. I’d specifically like to try a combination of Bayesian Sensors and the “Not so Binary” approach.

Hopefully I can come up with something which is both responsive to changes and also robust against false readings. I’ll be sure to write a post about that when I do. However, that’s it for now. Thanks for reading!

Continuous Integration/Deployment for Home Assistant with Gitlab CI

August 21, 2019Rob Connolly 2 Comments

This post may contain affiliate links. Please see the disclaimer for more information.

One of the best things about writing this blog is the interactions I have with other people. Of course it’s always nice to get feedback, whether it’s positive or (constructively) negative. It’s even better to see what similar projects other people are undertaking. Sometimes comments even start me off in a different direction than I had been taking.

This project was inspired by one such conversation. This started out rather gruff but actually ended up being really positive and motivated me to go further with an approach that I’d mostly dropped. The conversation in question was in relation to my recent “Seven Home Assistant Tips and Best Practices” post. Specifically it was around testing and deploying your Home Assistant config with Continuous Integration (via Gitlab CI).

I’m already familiar with Continuous Integration and Continuous Deployment through work. I had developed a minimal setup for validating my HASS config. However, I’d previously given up on it due to the slowness of Gitlab’s shared CI runners. What follows is my new attempt. Thanks to /u/automate_the_things and all the other commenters on that thread for the inspiration/persuasion to do this!

Setting Up a Local Runner

Ideally, I’d like to fully self host my own Gitlab instance. However, the recommended RAM requirements are between 4 and 8GB, which is a little ridiculous. Perhaps when I upgrade my server I’ll be able to spare enough RAM for this. For now I’m just running a local runner and connecting it to the cloud version of Gitlab.

I decided to go with deploying the runner as a Docker container and also executing my jobs also within their own containers. This fits with my recent Docker shenanigans and I’m familiar with this setup having deployed it for work. CI systems are one area where I’ve always felt that using containers makes sense, even before my recent Docker adventures. Each build running in it’s own container removes a lot of the complexity in managing build environments for different projects. It also means that your build machines only require Docker.

I set up my runner in a new VM on my main server. Then I pretty much just followed the official instructions to install the runner. I did convert the docker run command to a minimal docker-compose.yml file for ease of reproduction however:

version: '3'

services:
  gitlab-runner:
    image: gitlab/gitlab-runner
    volumes:
      - /home/rob/docker-data/gitlab-runner:/etc/gitlab-runner
      - /var/run/docker.sock:/var/run/docker.sock

Once that was done, I finished by getting the runner registered and connected to my project.

Build Pipeline Configuration

In my research for this project, I came across Frenck’s Gitlab CI configuration, which is really awesome (thanks @Frenck). I’ve based mine heavily on his with some tweaks for my configuration and environment. The finished pipeline runs the following jobs:

shellcheck – this job performs various checks on any shell scripts in the repository
yamllint – this job performs a full lint check of all my YAML files. Since I’ve never run this before it threw up loads of errors. I started fixing a few of these, but eventually marked the job with allow_failure: true to allow the pipeline to continue even with these errors. I’ll work on fixing these issues bit by bit over the next few weeks. I also remove some files which are encrypted with git-crypt
jsonlint – pretty much the same as yamllint, but for JSON files. Any files that are encrypted are excluded from this check.
markdownlint – similar as the previous jobs, but for checking of markdown files (such as the README.md file)
ha-latest – checks the HASS configuration against the current release of Home Assistant
ha-rc – runs a HASS configuration check against the next release candidate for Home Assistant
ha-dev – checks the HASS configuration against the development version of Home Assistant. Both of these jobs are configured to allow failure. This is just intended to give me advanced warning of and breaking configuration that may prevent HASS from starting up in a future release.
deploy – deploys the configuration to my HASS server. I’ll discuss this in more detail below.

You can find the finished configuration in my hass-config repository.

Deployment Approaches

There are several ways I could have done the deployment, which may suit different scenarios:

We could use the Home Assistant Gitlab CI sensor to poll the pipeline status. We would then trigger an automation to pull down the configuration when the pipeline passes. I haven’t tried this approach. However, it is potentially useful if your HASS server and Gitlab runner are on different networks and your HASS server is not publicly available.
We could use a pipeline webhook from Gitlab to a webhook handler in HASS. This would trigger an automation similar to that above. Again, I haven’t tried this. It would be useful if your Home Assistant instance and Gitlab CI runner are on different networks. However, it would require your HASS instance to be publicly available.
Similar to the approach above you could trigger the HASS webhook handler from a job in your Gitlab runner directly with CURL (rather than using the built in webhooks). This has the advantage over the previous two approaches that it gives you an explicit deploy stage in your pipeline. This is turn gives you the ability to track deployments via environments. It’s also potentially a lot simpler, since it would only be triggered if the previous stages in the pipeline passed. You also wouldn’t have to parse the JSON payload.
The approach I have taken is to deploy directly from the runner container via SSH. This is because my runner and HASS machines are running in the same network and so I can easily SSH between them without having to open any ports through my firewall. It also centralises all the deployment logic in the CI configuration, without any HASS automations needed.

My Deployment Job

As per my CI configuration, the deployment job is as follows:

deploy:
  stage: deploy
  image:
    name: alpine:latest
    entrypoint: [""]
  environment:
    name: home-assistant
  before_script:
    - apk --no-cache add curl openssh-client
    - echo "$DEPLOYMENT_SSH_KEY" > id_rsa
    - chmod 600 id_rsa
  script:
    - ssh -i id_rsa -o "StrictHostKeyChecking=no" $DEPLOYMENT_SSH_LOGIN "cd /mnt/docker-data/home-assistant && git fetch && git checkout $CI_COMMIT_SHA"
    - "curl -X POST -H \"Authorization: Bearer $DEPLOYMENT_HASS_TOKEN\" -H \"Content-Type: application/json\" $DEPLOYMENT_HASS_URL/api/services/homeassistant/restart"
  after_script:
    - rm id_rsa
  only:
    refs:
      - master
  tags:
    - hass

As you can see, this job runs in a plain Alpine Linux container and is deploying the the home-assistant environment. This allows me to track what versions were deployed and when from the Gitlab UI.

The before_script portion installs a couple of dependencies which we need for later and pulls the (password-less) SSH key we need for logging into the HASS server from the project variables. This is stored in the $DEPLOYMENT_SSH_KEY variable in the Gitlab configuration. The resulting file must have it’s permissions set to 600 to allow the SSH client to use it.

Moving on to the script portion. The first step performs the actual deployment of the repository to the server via SSH. Here we use our SSH key that we wrote out above. The public portion of this is installed on the HASS server as for the ci user. We also disable strict host key checking to prevent the SSH client prompting us to accept the fingerprint.

The SSH command connects to the server specified in $DEPLOYMENT_SSH_LOGIN, which is again set in the Gitlab variables configuration. This has the form ci@<hass host IP>. It should be noted here that the Alpine container defaults to using Google’s DNS. This means that resolving internal hostnames for your network will fail. I’m using the IP addresses for now to get around this.

Remote Control Commands

The SSH command sends a sequence of commands to be run on the HASS server. These commands are as follows:

cd /mnt/docker-data/home-assistant – change directory to the configuration directory on the server
git fetch – fetch all the new stuff via git
git checkout $CI_COMMIT_SHA – checkout the exact commit that we are running a pipeline for via one of Gitlab’s built in variables

This arrangement of commands allows me to control exactly what gets deployed to the server for each pipeline run. In this way we won’t accidentally deploy the wrong version if new code is checked into the server whilst our pipeline is running.

In order for the git fetch command to work another password-less SSH key is required. This time this is for the ci user on the HASS system. The public portion of this is installed as a deploy key for the project in Gitlab. I suppose it’s equally valid to pull changes via HTTPS (for public repos), but since the remote on my repository was already set up to use SSH I decided to continue using it.

Restarting Home Assistant from Gitlab CI

The second command in our script section is to restart Home Assistant after the configuration has been updated. Here we use CURL to call the homeassistant.restart service as per the API docs. The Home Assistant authentication token and URL are stored in Gitlab CI variables again.

Finally, we enter the after_script section, which will be executed even in the case that one of the above commands fails. Here we simply delete the id_rsa SSH key file.

I’ve restricted my deploy job to run only on pushes to the master branch. This allows me to use other branches in the repo as I please without having them deployed to the server accidentally. I’ve also used a tag of hass to prevent running on runners not intended for this job. Since I only have one runner right now this isn’t a concern, but having it in place makes things easier if/when I add more runners.

Conclusion

I’m really pleased with how this CI pipeline has turned out. However, I’m still a little concerned at how long all these steps take. The latest pipeline took 6 minutes and 42 seconds to run (plus the time it takes HASS to restart). This isn’t very much if it’s just a fire and forget operation. It is however, a long time if I am trying to iterate on my configuration. I’m going to play around with the runner configuration to see if I can get this down further. I also want to investigate options for testing on my local machine. In my previous attempt at this my builds could sit for longer than this waiting for one of Gitlab’s shared runners. So I’ve at least made progress in the speed department.

Next Steps

In terms of further improvements, I’d like better notifications of the pipeline progress and notifications when HASS competes it’s restart. I will implement these with Gotify. Right now I only get an email if the build fails from Gitlab. I’m also going to integrate the pipeline status into Home Assistant with the previously mentioned Gitlab CI sensor. I’m even tempted to turn one of my smart lights into a build light to alert me to problems!

I also want to take my use of CI in my infrastructure further. My next target will be building some modified Docker images for a couple of my services in CI. Deployment of the relevant Docker stacks is another thing I’d like to try out. I’ve not had chance to play with building containers via Gitlab CI before so that will be interesting.

I hope that you’ve enjoyed this post and that it’s inspired you to give something similar a go. Thanks to this pipeline I’ll never have to worry about HASS being unable to start due to a broken configuration again. I’m open to further improvements to this process. Please feel free to share any you may have via the feedback channels. Thanks for reading!

My Road To Docker – Part 2: My Home Automation Stack

July 16, 2019Rob Connolly 3 Comments

This post may contain affiliate links. Please see the disclaimer for more information.

This post is part of a series on this project. Here is the series so far:

In my first post of this series, I outlined my plan to convert my infrastructure over to a layered setup. This would consist of virtual machines (in various VLANs), with most of the services running in Docker. This post details the second stage of my road to Docker, although really was is the first stage since I’m writing these out of order! I actually converted my home automation systems over to Docker before tackling the web stack.

The motivation behind upgrading the home automation system first was to do it at the same time as I did a large update to Home Assistant, since I’d been holding back on updating. The main reason for this was the switch to Lovelace as the default UI, which I was dreading. As it turned out, I waited long enough for the awesome HASS developers to make all my problems go away (or at least the Lovelace related ones).

System Summary

I’ve written about my home automation setup before, but here is a brief recap of what I’m running (only the server side stuff):

Home Assistant: provides the main brains of the system and integrates all my devices as well as driving most of the automation
MQTT: via the Mosquitto broker, which is used to connect many of the devices in the house
Node-RED: the home automation server also houses the main Node-Red instance (others are scattered around on various Raspberry Pis). Although I call this the main instance, it doesn’t do a whole lot and mostly gets used for testing right now.
Zigbee2mqtt: this is a fairly recent addition and provides a bridge between my zigbee devices and MQTT/Home Assistant.

I had also been running InfluxDB and Grafana. However, something broke in my setup and I hadn’t got around to fixing it. I therefore decided to cut my losses with that and not reinstall it (for now).

Finding Docker Images

Luckily for me, the four main components of my system all have official/recommended Docker images available. This was useful as I’m always pretty reticent to use some questionably maintained image from the Docker Hub, mainly due to the lack of security updates. I also wanted to avoid building custom Docker images for now, until I work out a decent update strategy.

In addition to the four services above I wanted to run the ESPHome dashboard in order to manage my devices better. I had previously just been using the command line tool to build and upload to them. This also has an official Docker image.

I also ended up running a MaryTTS container to replace PicoTTS that I had been running for my voice announcements. This is due to the lack of PicoTTS inside the HASS Docker image. It was recommended that I use the silversniper/marytts image. Looking at this image, it hasn’t been updated in three years which. This reinforces my point about random images from Docker Hub. Luckily, this isn’t an externally facing application so it isn’t too critical from a security standpoint. However, I think I’ll look into updating that at some point.

Stacking Containers, again…

I set up a new clean VM inside my home automation VLAN. I was a little reserved about doing this, since it means everything in that VLAN (most of which is blocked from the internet) can see the full HA server. However, my main worry over not doing that was the mDNS used by ESPHome. If I can get Avahi/mDNS working across VLANs at some point I will move it. I still have a big network re-organisation to do, so hopefully it will get done then.

The full docker-compose.yml file for my new stack is given below:

version: '3'

services:
  mosquitto:
    image: eclipse-mosquitto
    restart: always
    ports:
      - 1883:1883
      - 8883:8883
      - 9001:9001
    volumes:
      - /mnt/docker-data/mosquitto/config:/mosquitto/config
      - /mnt/docker-data/mosquitto/data:/mosquitto/data
      - /mnt/docker-data/mosquitto/logs:/mosquitto/logs

  zigbee2mqtt:
    image: koenkk/zigbee2mqtt
    volumes:
      - /mnt/docker-data/zigbee2mqtt:/app/data
    devices:
      - /dev/ttyACM0:/dev/ttyACM0
    depends_on:
      - mosquitto
    restart: always
    network_mode: host

  homeassistant:
    image: homeassistant/home-assistant
    volumes:
      - /mnt/docker-data/home-assistant:/config
      - /etc/localtime:/etc/localtime:ro
    depends_on:
      - mosquitto
    restart: always
    network_mode: host

  nodered:
    image: nodered/node-red-docker:v8
    ports:
      - 1880:1880
    volumes:
      - /mnt/docker-data/node-red:/data
      - /etc/localtime:/etc/localtime:ro
    depends_on:
      - mosquitto
      - homeassistant
    restart: always

  esphome:
    image: esphome/esphome
    volumes:
      - /mnt/docker-data/esphome:/config
    restart: always
    network_mode: host

  marytts:
    image: sliversniper/marytts
    ports:
      - 59125:59125
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /mnt/docker-data/marytts/lib/voice-dfki-poppy-hsmm-5.2.jar:/marytts/marytts-5.1.2/lib/voice-dfki-poppy-hsmm-5.2.jar:ro
    restart: always

There’s nothing particularly earth shattering here. The main point of interest is that I mount the voice file for my preferred MaryTTS voice inside the container. Actually finding the voices is a little interesting. The official way to download them is a GUI tool that won’t run inside the container. I eventually found the XML file which lists all the available voices and extracted the URL of the one I wanted (the online demo helps to decide).

The only other parts worth noting are that I mount all my volumes under /mnt/docker-data, which is an NFS share onto the ZFS array of the virtual machine host. This then gets rolled into my normal backups. I also didn’t bother with a reverse proxy for any of this, since I already have one for HASS sitting in my DMZ (yet to be Dockerised). The other services just get accessed via the machine hostname and port since they are only used internally.

Sometimes Waiting Pays

I didn’t run into any issues particular to setting this up in Docker. At this point I think it’s a pretty well trodden path and this setup is pretty much standard. I did however run into several issues with upgrading to the latest Home Assistant.

First, lets tackle the elephant in the room – Lovelace. I was really worried going into this that it was going to be a huge amount of work. My mind was somewhat put at rest by seeing the UI editor in action via misperry’s video. When I actually came to it, the migration process automatically re-created my existing UI pretty much perfectly. Lesson learned: there is something to be said for waiting for mature software, rather than jumping on the new shiny thing immediately!

Lovelace itself is awesome! The ease of configuration has made me actually focus on making my HASS UI nicer rather than just the bare minimum I could get away with that I had previously. In the screenshot below, you can see my new “Outdoors” panel. This contains weather information, outdoor related sensor readings and a couple of local webcam views.

Remaining Issues

Most of my remaining issues were due to the Home Assistant “Great Migration”. This resulted in a load of entity IDs changing in various components. Obviously, this resulted in my having to update my configuration to change all the names. It took a little while to troubleshoot. This was because if the changed name is used in an automation, the automation has to actually fire to cause an error. In many cases it also just won’t fire, if the name is used in the triggers for the automation.

The final major issue I encountered, was with my frankly awesome vacuuming robot, which appeared to stop reacting to service calls in HASS. The underlying issue appears to be the Botvac D3 returning a different error message than the D7 that the library was tested with. So far this hasn’t been fixed, but I’m currently using the category: 2 workaround suggested and that’s working fine. I think I’ll have a look into fixing that issue and submit a PR when I get time.

Managing Updates

Managing updates to Docker images has always been an bit of an issue for me. In the past I’ve used Watchtower with some success. However, due to the capacity for breaking changes I want to manage HASS updates more carefully. It was suggested to me to just use a bash script which I can run periodically to do this. This isn’t something that had occurred to me before, probably because it’s so simple! Here’s the script I’m using:

#!/bin/bash
set -e

cd /mnt/docker-data/stacks/ha
sudo apt update
sudo apt upgrade -y
sudo apt autoremove -y
sudo apt clean
docker-compose pull
docker-compose down
docker-compose up -d
docker system prune -fa
docker volume prune -f
exit

This works beautifully and allows me to easily keep up with release to HASS and the other components, once I’ve verified that it’s reasonably safe to update.

Conclusion and Next Steps

Overall I’m pretty happy with how this move has turned out. Once the initial teething issues were all worked out the system has been very stable. I’m appreciating the extra utility of the ESPHome dashboard, which makes it very convenient to update my devices. It’s also great to be back on the latest version of Home Assistant.

In terms of next steps, I would like to give InfluxDB and Grafana another try. My main issue here has always been building the dashboards. It seems to be pretty tricky to get something both good looking and useful in Grafana. I also haven’t seen any pre-built dashboards for use with data from Home Assistant. Perhaps this is because they are so peculiar to individual setups.

I also have an LXD container running ZoneMinder. I’d like to re-deploy this as a Docker container on the same VM. Previously, I’ve not had too much luck running ZoneMinder in Docker. I’ll to see if the situation has improved when I tackle this migration.

I’m not actively working on any further migrations of other services to Docker at the moment, so there will probably be a break in this series for now. However, given my current success I’ll definitely be continuing on with this migration. I just want to work on some other projects for a while!

Self Hosting Update

May 15, 2018Rob Connolly 2 Comments

Since my first post on my self hosting setup, things have changed quite a bit. I thought I’d take the time to write up a few of those changes, having recently got much more interested in how I can improve my setup further. This has been stimulated at least in part by seeing some awesome setups browsing /r/homelab. There will be photos of the new setup at the bottom of this post.

So What’s Changed?

Well the first thing was that I moved house. This was a protracted move, with 4 months spent living at my parents place before moving into our new home. Due to space and other constraints I didn’t want to run the servers when living with them. Instead, I settled for playing around with a couple of Raspberry Pis in the mean time. One of these was a new Pi 3 bought specifically for the purpose of becoming a Kodi box. It does this quite nicely thanks to OSMC. The other was a Pi 2 which just had a testing setup of Home Assistant on for me to play around with.

Since moving into the new house, I’ve been building my self hosting setup back up. I think I’ve now surpassed level as I was at previously. Since everything had been offline for 4 months, I decided to make a clean break of things. After a back up I formatted the system drive of the main server and installed Ubuntu Server. This was with a view to running my services in LXD containers. This was made possible by the aforementioned Pi 3 becoming the main TV frontend, along with a Chromecast for Netflix duties. That meant the server could go fully headless for the first time and be relocated to the garage, where it can be attached to a noisy UPS.

What am I Running?

Currently, I’m running several containers on the server. These include:

A Home Assistant/Mosquitto/Node-RED container
A music server container running Mopidy+Snapcast for (eventually) multi-room audio
A Tvheadend container to replace Mythtv (not that I was unhappy with it, I just thought I’d try something new)
An Emby container for serving other media to Kodi (in future I’d like to add a second RPi/Kodi instance)
A CheckMK container to replace the previous built from source Nagios server
A couple of others for early stage testing of new projects

New Firewall

In addition to separating the main server from the media frontend I also invested in a new firewall box before moving into the new house. This was primarily due to the new house having a fibre connection. The USB Ethernet device on the old netbook I was using therefore became a bottleneck on Internet speed. I picked up one of those dual Ethernet Haswell based mini-computers from AliExpress.

This was originally running pfSense natively on the hardware. However, in order to try and get a little more out of the new hardware I’ve since swapped this out for a Proxmox host which runs pfSense in a VM (more on this in a future post). This runs really nicely and I’ve noticed that the case doesn’t get anywhere near as hot as it did running pfSense natively.Â Potentially this is confirmation bias on my part! The average air temperature has changed somewhat due to it getting towards winter.

I’m also running another VM on this system, which is hosting a testing install of Nextcloud. I haven’t transferred this to ‘production’ yet, mainly due to lack of time to get back to it. I’m pretty happy with it and will probably re-deploy it into an LXC container (Proxmox uses straight LXC not LXD) in order to reduce the memory footprint. I should have gone for more RAM in that box!. The main winner on the Proxmox install has been the ease with which I can do complex networking as required for the virtualised firewall and my VLAN setup. This is mainly due to the integration with OpenVSwitch, which I like a lot.

A Proper Switch

Having had the foresight to install Ethernet throughout our new home, I’ve needed to invest in a proper switch since we moved in. For a while I made do with piggy backing together my two wireless access points which provided 5 ports each. With this arrangement I was able to cover all the basics of my network. However, I wasn’t able to make every Ethernet jack in the house live and had no room for expansion.

I recently bought a TP-Link TL-SG1024DE 24 port switch. Whilst not the best switch in the world it is pretty good value for money and will serve my needs for the foreseeable future. Configuration of the VLANs is a little clunky, compared to the OpenWRT configuration interface I was using previously. However, everything works once it’s all configured. The great thing is I’ve been able to connect every port in the house as well as all my other gear and still have a ton of ports left over. The only feature I am missing in this switch is SNMP for monitoring, but I’m reasonably confident of being able to scrape the web interface at some point. Overall this switch is great for self hosting, where perhaps you don’t need those advanced features.

The Future

Based on my positive experience with Proxmox, I’ll probably migrate the main server to that at some point in the future. I’ve really enjoyed using LXD on Ubuntu, but Proxmox just seems better suited to my needs. The one feature I will miss from Ubuntu Server as a host is the kernel livepatching, which is really cool. The main thing holding me back from this at the moment is having to migrate all the existing LXD containers to LXC as there doesn’t seem to be a clean way to do this. This means the migration will have to wait until I can get all the services deployable via Ansible, which I’m working on.

Photos

As promised here are the photos of my self hosting setup. I’m using some standard garage shelving as a rack stand in. This works pretty well as I don’t have any rack mount gear except the new switch: