Lovelace Multi-room Audio Contols

Quick Project: Lovelace Multi-Room Audio Controller

This post may contain affiliate links. Please see the disclaimer for more information.

Welcome to a new segment on my blog! Here I’m going to do quick write ups of some of the little projects that I complete. These projects often come between the bigger ones that I usually write about. This is going to be bit of an experiment, but I’m aiming to publish these in addition to my usual blogging schedule. However, they will be done as time permits so may not be as regular.

I wanted to start doing posts like this, since I realised I do quite a few small projects which never make it onto the blog. Primarily this is because I just forget about them once they’re complete. Let’s see how it goes and please let me know if you like these posts in the comments or via Twitter.

Today’s Project: a Lovelace Multi-Room Audio Controller

This post is about a bit of playing around I was doing in Lovelace (the new Home Assistant UI) over the weekend. I started out installing the Home Assistant Community Store (HACS) and came across the Mini Media Player card. This struck me as the perfect thing for making better controls for my multi-room audio system. Without further ado, here are the finished Lovelace multi-room audio controller in all its glory:

Lovelace Multi-room Audio Contols
I love the beautifully minimalist look of this

Basically, we have controls of the main Mopidy music player. This is followed by the volume controls for the overall Snapcast group and the individual Snapcast clients. This is pretty much a copy of the layout in the Snapcast Android client. However, that lacks any way to control the player.

Get to the YAML!

Below is the YAML I used to create my Lovelace multi-room audio controller. Is should be noted that this can be entered through the GUI editor, by just switching to the raw editor. You don’t need to be using YAML mode.

This consists of a vertical stack card containing several mini media player cards with the group setting set to true to make them nest fine into a what appears visually to be a single control panel. I then just use the hide parameters to get rid of the controls I don’t need on each one and add a custom icon for each. Done!

Conclusion

I really like this layout. The only improvement I can think of would be if the cover artwork for the currently playing track was displayed. You can see I’ve tried this in the YAML above, but it didn’t work for me. This might be more of a limitation of the MPD integration in HASS than of mini media player.

That’s it for now. Again, please let me know if you like this post format and I’ll keep doing them in between my other posts.

If you liked this post and want to see more, please consider subscribing to the mailing list (below) or the RSS feed. You can also follow me on Twitter. If you want to show your appreciation, feel free to buy me a coffee.

Road to Docker Part 2

My Road To Docker – Part 2: My Home Automation Stack

This post may contain affiliate links. Please see the disclaimer for more information.

This post is part of a series on this project. Here is the series so far:


In my first post of this series, I outlined my plan to convert my infrastructure over to a layered setup. This would consist of virtual machines (in various VLANs), with most of the services running in Docker. This post details the second stage of my road to Docker, although really was is the first stage since I’m writing these out of order! I actually converted my home automation systems over to Docker before tackling the web stack.

The motivation behind upgrading the home automation system first was to do it at the same time as I did a large update to Home Assistant, since I’d been holding back on updating. The main reason for this was the switch to Lovelace as the default UI, which I was dreading. As it turned out, I waited long enough for the awesome HASS developers to make all my problems go away (or at least the Lovelace related ones).

System Summary

I’ve written about my home automation setup before, but here is a brief recap of what I’m running (only the server side stuff):

I had also been running InfluxDB and Grafana. However, something broke in my setup and I hadn’t got around to fixing it. I therefore decided to cut my losses with that and not reinstall it (for now).

Finding Docker Images

Luckily for me, the four main components of my system all have official/recommended Docker images available. This was useful as I’m always pretty reticent to use some questionably maintained image from the Docker Hub, mainly due to the lack of security updates. I also wanted to avoid building custom Docker images for now, until I work out a decent update strategy.

In addition to the four services above I wanted to run the ESPHome dashboard in order to manage my devices better. I had previously just been using the command line tool to build and upload to them. This also has an official Docker image.

Road to Docker Part 2
Looks like I have a few devices to update!

I also ended up running a MaryTTS container to replace PicoTTS that I had been running for my voice announcements. This is due to the lack of PicoTTS inside the HASS Docker image. It was recommended that I use the silversniper/marytts image. Looking at this image, it hasn’t been updated in three years which. This reinforces my point about random images from Docker Hub. Luckily, this isn’t an externally facing application so it isn’t too critical from a security standpoint. However, I think I’ll look into updating that at some point.

Stacking Containers, again…

I set up a new clean VM inside my home automation VLAN. I was a little reserved about doing this, since it means everything in that VLAN (most of which is blocked from the internet) can see the full HA server. However, my main worry over not doing that was the mDNS used by ESPHome. If I can get Avahi/mDNS working across VLANs at some point I will move it. I still have a big network re-organisation to do, so hopefully it will get done then.

The full docker-compose.yml file for my new stack is given below:

There’s nothing particularly earth shattering here. The main point of interest is that I mount the voice file for my preferred MaryTTS voice inside the container. Actually finding the voices is a little interesting. The official way to download them is a GUI tool that won’t run inside the container. I eventually found the XML file which lists all the available voices and extracted the URL of the one I wanted (the online demo helps to decide).

The only other parts worth noting are that I mount all my volumes under /mnt/docker-data, which is an NFS share onto the ZFS array of the virtual machine host. This then gets rolled into my normal backups. I also didn’t bother with a reverse proxy for any of this, since I already have one for HASS sitting in my DMZ (yet to be Dockerised). The other services just get accessed via the machine hostname and port since they are only used internally.

Sometimes Waiting Pays

I didn’t run into any issues particular to setting this up in Docker. At this point I think it’s a pretty well trodden path and this setup is pretty much standard. I did however run into several issues with upgrading to the latest Home Assistant.

First, lets tackle the elephant in the room – Lovelace. I was really worried going into this that it was going to be a huge amount of work. My mind was somewhat put at rest by seeing the UI editor in action via misperry’s video. When I actually came to it, the migration process automatically re-created my existing UI pretty much perfectly. Lesson learned: there is something to be said for waiting for mature software, rather than jumping on the new shiny thing immediately!

Lovelace itself is awesome! The ease of configuration has made me actually focus on making my HASS UI nicer rather than just the bare minimum I could get away with that I had previously. In the screenshot below, you can see my new “Outdoors” panel. This contains weather information, outdoor related sensor readings and a couple of local webcam views.

Road to Docker Part 2
I probably should have taken this screenshot during the day

Remaining Issues

Most of my remaining issues were due to the Home Assistant “Great Migration”. This resulted in a load of entity IDs changing in various components. Obviously, this resulted in my having to update my configuration to change all the names. It took a little while to troubleshoot. This was because if the changed name is used in an automation, the automation has to actually fire to cause an error. In many cases it also just won’t fire, if the name is used in the triggers for the automation.

The final major issue I encountered, was with my frankly awesome vacuuming robot, which appeared to stop reacting to service calls in HASS. The underlying issue appears to be the Botvac D3 returning a different error message than the D7 that the library was tested with. So far this hasn’t been fixed, but I’m currently using the category: 2 workaround suggested and that’s working fine. I think I’ll have a look into fixing that issue and submit a PR when I get time.

Managing Updates

Managing updates to Docker images has always been an bit of an issue for me. In the past I’ve used Watchtower with some success. However, due to the capacity for breaking changes I want to manage HASS updates more carefully. It was suggested to me to just use a bash script which I can run periodically to do this. This isn’t something that had occurred to me before, probably because it’s so simple! Here’s the script I’m using:

This works beautifully and allows me to easily keep up with release to HASS and the other components, once I’ve verified that it’s reasonably safe to update.

Conclusion and Next Steps

Overall I’m pretty happy with how this move has turned out. Once the initial teething issues were all worked out the system has been very stable. I’m appreciating the extra utility of the ESPHome dashboard, which makes it very convenient to update my devices. It’s also great to be back on the latest version of Home Assistant.

In terms of next steps, I would like to give InfluxDB and Grafana another try. My main issue here has always been building the dashboards. It seems to be pretty tricky to get something both good looking and useful in Grafana. I also haven’t seen any pre-built dashboards for use with data from Home Assistant. Perhaps this is because they are so peculiar to individual setups.

I also have an LXD container running ZoneMinder. I’d like to re-deploy this as a Docker container on the same VM. Previously, I’ve not had too much luck running ZoneMinder in Docker. I’ll to see if the situation has improved when I tackle this migration.

I’m not actively working on any further migrations of other services to Docker at the moment, so there will probably be a break in this series for now. However, given my current success I’ll definitely be continuing on with this migration. I just want to work on some other projects for a while!

If you liked this post and want to see more, please consider subscribing to the mailing list (below) or the RSS feed. You can also follow me on Twitter. If you want to show your appreciation, feel free to buy me a coffee.

Multi-Room Audio Client

Multi-Room Audio System: Indoor and Outdoor Audio with Snapcast and Mopidy

This post may contain affiliate links. Please see the disclaimer for more information.

One of the projects I really wanted to do when moving into our new house was build a multi-room audio system. Traditional multi-room audio systems, such as Sonos, cost a massive amount for the functionality they provide. It looks like a cheaper alternative is now available using Chromecasts, but you are still at the mercy of what the manufacturer wants to do (like discontinuing the Chromecast Audio).

In this post I’m going to detail my multi-room audio setup, which plays perfectly synced audio across three sets of speakers, both indoors and outdoors. This system is 100% DIY and uses Free Software throughout. It’s also cheaper than even a single Sonos speaker.

System Overview

My system is comprised of a central server running Mopidy and Snapcast (the snapserver portion) and three audio players, each running the Snapcast client (snapclient).

Two of the clients are resident on the Raspberry Pi systems we use for Kodi on our TVs. One of these is located in the Living Room and connects into our soundbar. The other is located in the master bedroom and currently just uses the TV speakers.

The third client is located on a Raspberry Pi in our loft space, which is connected via an amplifier to speakers mounted outdoors by our patio.

The parts list for this setup is as follows:

The Fusion speakers listed above are outdoor/marine rated and certainly seem fine in the New Zealand climate (warm humid summers, wet cool winters). They are definitely not the best speakers in the world (the price reflects that). However, the quality is sufficient for my application of background/work music in an outdoor environment.

Overall, the total cost for the components ordered for this project was less than NZ$250. This comes in at less than the price of a single Sonos speaker. I’ve not included the Raspberry Pis in this, since I already had them and only one was specifically installed for this project.

Software Setup

The software setup is a pretty standard for this kind of project – basically just Mopidy feeding audio to Snapcast. As such I’m not going to give a full installation guide, since there are plenty of resources available. Take a look at the links below for full instructions (these are the resources I used when setting this up):

Multi-Room Audio UI
The Web UI Via Iris

In terms of client/remote control software, I’m using Iris as a web interface for Mopidy. On the Android side I’m using M.A.L.P. as well as the Snapcast app. M.A.L.P. seems to be a reasonable MPD client and supports multiple servers, which may come in useful in future. The main issue I have with it is that it gets the album art wrong frequently and there seems to be no way to override it’s choices (or use the correct album art from the server).

Of course, I also have both Mopidy and Snapcast integrated with Home Assistant!

Outdoor Speaker Hardware Setup

So far, so easy. Here is where I ran into issues. I mounted the speakers to the brick wall of our house just fine, but ran into problems running the cables up through the roof space to the amplifier. This was mainly due to one speaker being on the corner of the house where the roof is low. In this corner the steel supports for the roof were too close together for me to squeeze through. Also the level of the soffit where the cable came in was lower than ceiling height, so that the soffit forms a well around the outside of the house. All this made it nearly impossible to grab the cable.

Multi-Room Audio Speakers
Left Speaker (the one with the tricky cable)
Multi-Room Audio Speakers
Right Speaker

I fashioned a makeshift tool from an old mop handle and reacher grabber with a line attached to the handle so that I could actuate it from the end of the pole. I even went as far as installing the Android IP Webcam app on an old phone and mounting that on the far end. With this I could then view the image on my phone and use the light on the camera end to see better. This helped, until the battery on the phone died! Eventually I managed to grab the cable by pushing the whole length of it up through the soffit. The resulting bundle was much easier to grab.

Phew, now we’re getting somewhere…

Overall, getting the speakers installed took most of a day, with several hours spent laying on my front in the (hot) loft space trying to grab the cable. The provided speaker cables also had to be lengthened with some extra speaker cable from my local DIY store. Luckily I knew this before I installed them and didn’t have to pull them back.

Multi-Room Audio Client
The loft RPi with relays, sitting on top of the amplifier
Multi-Room Audio Client
The LED on the amplifier is unnecessarily bright

The remainder of the install was pretty much plug and play. I connected one of the USB soundcards to the Raspberry Pi and connected it’s output via audio cable to the amplifier. I spliced the relay into the 12V power line from the power supply to the amplifier to allow me to remotely control it’s power. Both the RPi and the amp are powered from the mains sockets I previously had installed in the loft.

Node-RED Relay Control

Multi-Room Audio Power Control
Power Control is done via Home Assistant

As with the relay power control for my room sensors, I used Node-RED to turn the relay on and off via MQTT. The flow uses my Home Assistant MQTT Discovery approach to be automatically added to HASS. There’s not much to say about this since it’s pretty much identical to the setup for the room sensors. Here’s the flow:

Multi-Room Audio Power Control
The relay power control flow

And here’s the corresponding JSON:

I also have a couple of automations which I use to mute/unmute the relevant Snapclient when the speakers are turned off. My completely unfounded hypothesis is that Snapcast should be intelligent enough to not send any data to muted clients, which should reduce unnecessary traffic on the network. I’ve not done any investigation to verify this however. In any case, here are the automations:

Indoor Setup with Libreelec and Kodi

It wouldn’t be a multi-room audio setup with out multiple clients! So on to the indoor systems. These are the running on my two Libreelec systems, connected to the TVs. The first of these is the most interesting since that connects to to our Polk Signa S2 soundbar. I’m actually planning a review of this in the near future, but for now we’ll just say it sounds awesome. I didn’t include it in the hardware list above since I didn’t purchase it just for this project.

I connected to the soundbar using the second USB soundcard and audio cable. This means I can play audio without having the TV on, just by setting the soundbar to it’s AUX input. The other system in the master bedroom, just sends audio via the HDMI port to the TV.

Multi-Room Audio on Libreelec
Settings of the Libreelec Snapclient Plug-in

On the software side of these I used the excellent Libreelec Snapclient plug-in. Just install it from the official Libreelec repo and you’re good to go. In order for the auto-discovery to work, you should make sure that the Snapserver and Libreelec machine are on the same network. The only other issue I had is that sometimes the ‘list sound cards’ dialog in the plugin settings wouldn’t work. I found it easier to just list the devices on the command line with snapclient -l and put the relevant device number into the addon settings.

Conclusion and Next Steps

Overall, this system is pretty great. It’s served us well for outdoor audio all through the summer and has become our primary way of listening to our music collection. There are a few rough edges, like the issues with album art on Android.

The main other point of complaint is the profusion of volume controls. This could be a separate rant altogether, since everything has it’s own volume control for some reason. For this system I just don’t touch the volume in Mopidy and use the individual channel controls in Snapcast. It looks like there is now a plugin to provide better integration here, but I haven’t tried it yet.

The next steps for this system will be to re-build the server side system as part of my ongoing migration to Docker+VMs. At this point I’d like to add a couple more groups to the Snapserver. One of these will be for audio streaming in over Bluetooth. This will allow for us to stream audio directly from our phones to any of the speakers in the house.

The second group will be for TTS notifications from Home Assistant. I know I can move channels between groups via HASS automations to decide where the audio goes. The main stumbling block on this at the moment is how to get the audio from the HASS server to the media server, which will be separate VMs. If anyone has any ideas here, please let me know!

That’s it for now. Thanks for reading!

If you liked this post and want to see more, please consider subscribing to the mailing list (below) or the RSS feed. You can also follow me on Twitter. If you want to show your appreciation, feel free to buy me a coffee.


Traffic for the last 30 days

Site Update: Where We Are and Plans For The Future

This post may contain affiliate links. Please see the disclaimer for more information.

Regular readers may have noticed some changes around here of late. Certainly, I’ve been giving the site more love. It’s been moved to a new host and I’ve been playing around with the layout. You’ll also notice from the message above that I’m experimenting with affiliate links. The latest addition is some (hopefully minimal) AdSense ads.

This is all part of my new longer term plan to turn this site into something more than just a place to document my projects. I want to build the site both into a resource for the community and into something that can hopefully make some money to support me doing more projects.

A Change in Perspective

In turn, the plan for this site is part of a larger change in perspective that I’ve had this year. For a long time, I’ve wanted to start my own business. This is something I’ve always been open with – even with my employers. This desire wasn’t driven by wanting to make tons of money, I just wanted the freedom to set my own agenda. The problem was I could never come up with an idea compelling enough for me to want to lay aside other projects in order to work on it.

As time went on, I realised what I actually want is to get paid to work on my own projects (actually I’d happily do it for free if I didn’t need money to live). The problem is no-one is going to pay me to do that! By building up this blog into something that brings in some income (however modest), I will actually be achieving that goal. I may never be able to live off the proceeds, but having the exposure will hopefully open up other opportunities.

My Ultimate Goal

The explanation for why this has come to the front of my mind this year lies in my discovery of the Financial Independence (A.K.A FIRE) movement. This is my ultimate goal. Unfortunately, I’m a long way away from that. It would have helped if I’d discovered it 10 years ago. However, 20’s me didn’t really care about money beyond having enough to live comfortably and not being in debt.

The main tenet of FI is that having enough money set aside to support your (hopefully reasonably minimal) lifestyle buys you the freedom to spend your time as you like. It’s really not about the money. Money is just the means to that end.

If only this guy had known about Financial Independence

FI is a long term game. However, for me there would be no functional difference between being fully financially independent and getting paid to work on projects and document them on this site (aside from the need to keep doing it, to ensure a steady income). So with that in mind, it would be nice if I could eventually make enough from this blog to be able to do it full time.

In the meantime, I’m intending to use any proceeds from this blog (after hosting costs), to both finance future projects and to put towards the FI fund.

Wait a minute, did you say AdSense?

As in Google AdSense. Yes, I did. However, I didn’t come to this decision lightly.

This is a blog about self hosting, DIY electronics and Home Automation. Google really doesn’t fit in around here. I don’t like the tracking inherent in Google’s Ad platform and I don’t like being dependent on an external service to serve content on my site.

I’ve thought long and hard about whether I would run ads at all, never mind Google ones. I’ve also investigated other Ad networks and specifically talked to someone at Mediavine. Unfortunately, not only do I not yet meet their traffic requirements, they also require the use of Google Analytics, so I wouldn’t be any freer from Google.

Ideally, I’d like to host my own ads (if I have to have ads at all), for products that are actually useful and relevant. As a (very) small publisher this seems to be out of reach right now. However, if you’d like to advertise here, feel free to get in contact!

So Adsense is here for now. I’m treating the period of the next few months as a trial to see how well it does. I’ll be sure to report back on whether I think it’s worth it. Given the content of this site, I’m assuming that anyone in my audience who doesn’t like the ads has the technical chops to block them. I don’t mind this, my content is still free for you. However, please don’t complain about it unless you’re willing to buy me a coffee! 🙂

Current Status

To have any chance of success in my efforts, I need to bring more traffic in to the site. The main way to do this is of course to produce more content on a more regular basis. This is something I’ve struggled with in the past. Although I’m improving here, I still have a long way to go.

Traffic for the last 30 days
Site traffic for the last 30 days

In order to benchmark to my progress, I’m publishing some of my traffic stats. As you can see from the graph above, I’m getting some reasonably decent peaks when I publish an article, with massive drops in between.

Over the time period above:

  • There were approximately 3500 visits
  • Maximum daily traffic was 886 visits (corresponding to my last Home Assistant article)
  • Minimum daily traffic was 10 visits, but on non posting days is usually somewhere in the 20s or 30s

I’m not going to bother with any further breakdown, such as traffic sources, etc. That can get really boring!

Where to Next?

My immediate goals are what I’m calling the “100-100-100 Challenge”, these are as follows:

  • Achieve an average of 100 visits per day on non-posting days (with a proportionate increase on posting days)
  • Gain 100 followers on Twitter (I’m actually 3/4 of the way there!)
  • Earn $100 (NZ) in blog related income (after fees and tax)

These seem fairly modest and achievable. In addition I’m trying to set up a blogging schedule so that I can produce one post a week – which will certainly help with the above goals.

That’s pretty much it for this update. As always, if you have any feedback on my plans or on the changes to the site, please leave a comment or get in contact.

If you liked this post and want to see more, please consider subscribing to the mailing list (below) or the RSS feed. You can also follow me on Twitter. If you want to show your appreciation, feel free to buy me a coffee.

My Road to Docker: Traefik

My Road to Docker – Part 1: My Web Stack

This post may contain affiliate links. Please see the disclaimer for more information.

This post is part of a series on this project. Here is the series so far:


I’ve been following and using Docker since it’s early days. I think 0.6 was one of the first releases I tried. That said, I’ve never actually had that much luck getting it deployed on meaningful parts of my infrastructure. Usually, I would start some new deployment of something with the intention of using Docker. Then I’d hit some issue which made it difficult (or at least more difficult than it should be). Either that, or I’d run some Docker containers as a testing deployment for a while and then pull them down. It’s safe to say my road to Docker has been long, winding and full of potholes!

Recently, I’ve been giving my infrastructure choices more thought. My main goals are that I want to reduce maintenance and keep the high level of separation and security I have between my different services. As detailed in my last post on my self hosting setup, I’ve been using LXD fairly successfully. The problem with this is really that LXD containers are more like virtual machines. Each container is a little Linux system which needs to be kept up to date, monitored and generally looked after. When you have one container per service this becomes a chore. In actuality I don’t have one LXD container per service. Instead I group dependent services such as databases with their parent. However, I’ve still ended up with quite a few containers.

Decisions, decisions…

I really like the separation I’m able to have by keeping my LXD containers on separate VLANs. Docker does support this via macvlan, but the last time I played with it I had to manually assign IPs to each container. It looks like Docker will now do this automatically. However, it allocates from it’s own IP pool and won’t use your DHCP server. This means I can’t assign IPs from my Pfsense firewall. Also having one public (LAN) IP per Docker service kinda sucks and I also don’t want to rely on just one project for the security of my system.

Additionally, I have the need to run at least one VM anyway for my virtualised Pfsense firewall. This brings me to the idea of running Docker inside VMs. Each VM can be assigned to the relevant VLAN interface and get it’s IP from DHCP. I also get the benefit of two levels of isolation using different technologies. I haven’t yet decided on whether to use Proxmox or Ubuntu Server+Libvirt+Cockpit for the host systems. Hopefully this will be the subject of a future post.

On To Today

All the above is pretty much background, because I actually don’t want to talk about my locally hosted stuff today. Instead I’m going to talk about my stuff in the cloud – specifically the hosting for this site.

I’ve used Linode since around 2011 when I set up my self hosted mail server. Since this time, I’ve hosted this site on a fairly standard LAMP setup on that same server. The mail server install is now getting on a bit, but being CentOS based it’s all still supported for updates until next year. However, in preparation for re-building the mail server I decided to move the web stuff out onto another server and run it in Docker. This basically gives me the same setup of Docker-on-VM as I have on my local infrastructure. Just the hypervisor UI differs – it’s all KVM underneath.

Since I was moving the site to a new server, I also decided to move closer to my audience. According to my Matomo statistics this is predominantly US based. To this end I span up a new $5 Linode (Nanode), running Ubuntu 18.04 in their New Jersey datacenter. This should also be pretty fast for European visitors (to be honest it’s lightning fast even from NZ).

Getting Going

After doing some pretty standard first 10 minutes on a server stuff, I set about installing Docker. Instead of installing via my usual method of adding the APT repository I decided to see how installing from a Snap would work in production:

A few seconds later Docker was installed. However, I ran into a wrinkle when trying to add myself to the Docker group. Basically, it wouldn’t allow me to run Docker commands without root access (still). The solution is to add the Docker group before installing the Snap, so I removed it, added the group and reinstalled. So full instructions for installing Docker via Snap should more accurately be:

Stacking Containers

I’d resolved to install the blog inside the official WordPress container and connect use mariadb for the database. Since I was moving my install over, it was slightly more complex than it would be for a clean install, but this article put me on the right track.

The main issue I encountered didn’t happen until I had the site running and was trying to get HTTPS running. Due to Let’s Encrypt’s HTTP verification this had to be done after the DNS settings were updated i.e. the site was live. This issue manifested as WordPress serving HTTP URLs for all links and embedded content. This leads to mixed content warnings/blocks in Firefox and Chrome. The issue seems to be common to any setup where HTTPS is handled by a reverse proxy. Basically, in this setup WordPress isn’t aware that it should be using HTTPS and needs to be told. Adding the following to wp-config.php fixes the problem:

I decided to use Traefik as my reverse proxy, which I’m quite pleased with. The automated service discovery is pretty awesome and will really come into it’s own on my other self hosted infrastructure. I also whacked a Varnish cache (from this Docker image) in between. So far I haven’t done much with Varnish, but it’s there for when I get time to tweak it.

Moving Matomo

I also moved my Matomo Analytics instance to the new server using the official Docker image. This gave me much less trouble than WordPress since I just allowed it to use the embedded Matomo version with my configuration file and database from the old install.

I connected Matomo up directly to my Traefik instance, without running it through Varnish, to avoid any potential issues with the cache interfering with my statistics. Traefik just worked with this setup, which was pretty refreshing.

My Road to Docker: Traefik
My Traefik Dashboard

PSA: Docker will Ignore Your Firewall!

Along the way, I ran into another issue. Before I changed the DNS settings to make the site live, I was binding Traefik to port 8080 and accessing it via an SSH tunnel for making changes in WordPress. Since I had configured UFW to block everything (except SSH) when I set up the server, I thought this was a nice secure setup for debugging.

I was wrong.

I only noticed something was off because I happened to have the log output from Traefik open and saw random IPs in the logs. Luckily, the only virtual host I had configured at that point was localhost:8080 so all the unwanted visitors got 404 responses. Needless to say I pulled down all the containers until I could work out what was going on.

This appears to be a known interaction between Docker and any firewall utility (including both UFW and Firewalld). The issue is inherent in the way that Docker uses iptables to route traffic to your containers. Basically, the port forwarding rules go in the NAT chain in iptables. This means incoming traffic is re-routed before it hits the INPUT chain containing the rules from UFW/Firewalld.

I tried the fix suggested – disabling iptables support in Docker, but this completely broke inter-container connectivity (unsurprising, when you think about it). My solution for now is just to be really careful when binding ports. Make sure that any ports you don’t want to give access to the outside world are bound only to 127.0.0.1. There is also the DOCKER-USER iptables chain if you need more flexibility, but it means you need to use raw iptables rules.

This issue is a major security flaw and needs to be given more attention. Breaking administrator expectations of security like this is going to lead to loads of services being exposed to the big bad Internet that really shouldn’t be!

My Full Web Stack

Below is the docker-compose.yml file for my full web stack. I store any secret variables in an env.sh file which I source before running my docker-compose commands.

This is all pretty much as described, however there are two notable points:

  • I went for separate mariadb containers for the databases for WordPress and Matomo. This is because the official mariadb container only supports creation of one database/user via environment variables. I’m not super happy with this arrangement, but it is working well and doesn’t seem to use too much memory.
  • The aliases portion under the network configuration for the Traefik container allow the other containers to route internal requests back to themselves. This helps with things such as the loopback check in WordPress, which will otherwise fail.

That’s pretty much all there is to it!

Conclusion

Overall, I’m really happy with the way this migration has worked out. I’ve even been able to downgrade the Linode plan for the original server to the $5/month plan, since it now has less load. This means I’m paying the same as I was for the single server for two servers, although with half the RAM each. I think I paid an extra $2-3 during the migration period, since that took me a while to compete. Even that isn’t too bad.

I’ve already started on further Docker migrations on some of the infrastructure I have on my home servers. These should be the subject of further posts.

If you liked this post and want to see more, please consider subscribing to the mailing list (below) or the RSS feed. You can also follow me on Twitter. If you want to show your appreciation, feel free to buy me a coffee.