getting started with appdaemon

Getting Started with AppDaemon for Home Assistant

This post may contain affiliate links. Please see the disclaimer for more information.

Continuing on from last weeks post, I was also recently persuaded to try out AppDaemon for Home Assistant (again). I have previously tried out AppDaemon, so that I could use the excellent OccuSim app. I never got as far as writing any apps of my own and I hadn’t reinstalled it in my latest HASS migration. This post is going to detail my first steps in getting started with AppDaemon more seriously.

I should probably start with a run down of what AppDaemon is for anyone that doesn’t know. The AppDaemon website provides a high level description:

AppDaemon is a loosely coupled, multithreaded, sandboxed python execution environment for writing automation apps for home automation projects, and any environment that requires a robust event driven architecture.

In plain English, that basically means it provides an environment for writing home automation rules in Python. The supported home automation platforms are Home Assistant and plain MQTT. For HASS, this forms an alternative to both the built in YAML automation functionality and 3rd party systems such as Node-RED. Since AppDaemon is Python based, it also opens up the entirety of the Python ecosystem for use in your automations.

AppDaemon also provides dashboarding functionality (known as HADashboard). I’ve decided not to use this for now because I currently have no use for it. I also think the dashboards look a little dated next to the shiny new HASS Lovelace UI.

Installation

I installed AppDaemon via Docker by following the tutorial. The install went pretty much as expected. However, I had to clean up the config files from my old install before proceeding. The documentation doesn’t provide an example docker-compose configuration, so here’s mine:

I’ve linked the AppDaemon container to an internal network, on which I’ve also placed my HomeAssistant instance. That way AppDaemon can talk to HASS pretty easily.

You’ll note that I’m not passing any environment variables as per the documentation. This is because my configuration is passed only via the appdaemon.yaml file, since it allows me to use secrets:

You’ll see here that I use the docker0 interface IP to connect to HASS. I tried using the internal hostname (which should be homeassistant on my setup), but it didn’t seem to work. I think this is due to the HASS container being configured with host networking.

Writing My First App

I wanted to test of the capabilities and ease of use of AppDaemon. So, I decided to convert one of my existing automations into app form. I chose my bathroom motion light automation, because it’s reasonably complex but simple enough to complete quickly.

I started out by copying the motion light example from the tutorial. Then I updated it to take configuration parameters for the motion sensor, light and off timeout:

I’ve also added a couple of utility methods to manage the timer better and also to specify more complex logic to restrict when the light will come on. Encapsulating both of these in their own methods will allow re-use of them later on.

The timer logic of the example app is particularly problematic in the case of multiple motion events. In the original logic one timer will be set for each motion event. This leads to the light being turned off even if there is still motion in the room. It also caused some general flickering of the light between motion events and triggered callbacks. I mitigate this in the set_timer method here by first cancelling the timer if it is active before starting a new timer with the full timeout.

At this point, we have a fully functional and re-usable motion activated light. We can instantiate as many of these as we would like in our apps/apps.yaml file, like so:

Note that we haven’t yet recreated the functionality of my original automation. In that automation, the brightness was controlled by the door state. We’ll tackle this next.

Extending the App

Since our previous MotionLight app is just a Python object, we can take advantage of the object orientated capabilities of the Python language to extend it with further functionality. Doing so allows us to maintain the original behaviour for some instances, whilst also customising for more complex functionality.

Our subclassed light looks like this:

Here we can see that the initialize method loads only the new configuration parameters. The existing parameters from the parent class are loaded from the parent’s initialize method via the super call. The new configuration options are passed as lists, allowing us to specify several bedroom or other doors. In order to set the relevant callbacks I loop over each list and set the callback. The callback is the same for each entry in the list since it only matters what type they are. The specifics of each door are irrelevant.

Next we have the actual callback methods for when the doors open. These just set the internal variable last_door to the relevant value and log it for debugging purposes.

Most of the new logic comes in the motion_callback method. Here I have reused the is_light_times and set_timer methods from the parent class. The remainder of the logic first checks that the light is off and then recreates the operation of the template I used in my original automation. This sets the light to dim if the last door opened was to one of the bedrooms and bright otherwise. There are also some time based restrictions on this for times when I always want the light bright.

The configuration is pretty similar to the previous example, with the addition of the lists for the doors:

Conclusion and Next Steps

The previous automation (or more rightly set of automations) totaled to 78 lines. The Python code for the app is only 56 lines long. However, there is another 11 lines of configuration required. By this measurement, it seems like the two are similar in complexity. However, we now have an easily re-usable implementation for two types of motion controlled lights with the AppDaemon implementation. Further instances can be called into being with only a few lines of simple configuration. Whereas the YAML automation would need to be duplicated wholesale and tweaked to fit.

This power makes me keen to continue with AppDaemon. I’m also keen to integrate it with my CI Pipeline. Although I’m actually thinking of separating it out from my HASS configuration. With this I’d like to try out some more modern Python development tooling, since it’s been quite some time since I’ve had the opportunity to do any serious Python development.

I hope you’ve enjoyed reading this post. For anyone already using AppDaemon, this isn’t anything groundbreaking. However, for those who haven’t tried it or are on the fence, I’d highly recommend you give it a go. Please feel free to show off anything you’ve made in the feedback channels for this post!

If you liked this post and want to see more, please consider subscribing to the mailing list (below) or the RSS feed. You can also follow me on Twitter. If you want to show your appreciation, feel free to buy me a coffee.

my road to docker smtp

My Road to Docker: Sorting Out SMTP

This post may contain affiliate links. Please see the disclaimer for more information.

This post is part of a series on this project. Here is the series so far:


Having said in my last ‘Road to Docker’ post that I didn’t have any current plans for another post, something came up which warrants a write up here. I managed to solve the problem in question in a Dockerised fashion and I’m quite pleased with the solution. Let’s get into it….

The Backstory

After converting my web stack over to Docker recently, I had installed the WP Mail SMTP plugin in WordPress to handle mail from the site. This was required since WordPress could no longer send via a local mail setup. I configured the plugin to send via my existing mail server. This worked well for a while – then I encountered a speed bump on my road to Docker!

For some reason (I think due to an update of the WordPress container), I just stopped receiving email from the site. Upon investigation it seemed that the TLS connection was unable to be started correctly. I got the following debug log when testing mail via WordPress:

Looking into the logs on the mail server, I found the following corresponding error:

I tried googling the error, but after trying a few of the suggested fixes, I gave up and decided to solve the problem a different way.

Fixing it, Take 1

My first attempt involved installing Postfix on the host in a smarthost configuration to my main server. This is the same setup as I use on most of my servers for system mail from cron, etc (via a custom Ansible role).

After getting the mail system running and able to send mail from the host, I tried to configure it in WordPress. However, I was unable to connect to the host machine from the container on the Docker host IP address. I investigated this and found that because I was using a private network the IP address was different than the standard Docker interface address.

Trying this address didn’t work either. Perhaps this is a security feature in Docker, or perhaps I was doing it wrong. Either way it pushed me on to a better solution.

Fixing it, Take 2

My next plan involved putting Postfix into a container. This would be put on the same private network as the WordPress container to allow access. I needed to keep the smarthost configuration to talk to the main mailserver. A quick search turned up a suitable image in the form of boky/postfix. This image is intended exactly for this purpose and I was able to set it up without too much trouble.

To spin this up I added the following to my previous docker-compose.yml file:

Pretty simple! As per the previous post I put all the secrets into an env.sh file to keep them separate from the stack.

The mail forwarder is available both on the internal Docker network and on the host system to replace the native mail forwarding setup. Setting this up with WordPress ended up being trivial (see screenshot below). However, I required some further configuration to make the mail on the host system work.

my road to docker smtp
We just use the container name as the hostname and 587 as the port! Authentication and SMTP aren’t required for the local connection.

MSMTP Setup

In order to redirect the system mail from the host via the Dockerised mail forwarder, I had to set up MSMTP. This is pretty effectively documented elsewhere, so I won’t go into details. The only differences in this setup are that we don’t require authentication or TLS to the mail forwarder because it’s only available locally. The mail forwarder itself is already handling authentication and TLS to the main mail server.

For reference, here is the msmtprc file I ended up with:

Conclusion

Here we’ve seen how to quickly deploy a mail forwarding server for your Dockerised applications. We’ve also configured our host system to also use it, so we don’t need to run two forwarders.

I feel that this is the best solution to this problem (although I still don’t quite know what the original problem was!). It feels like a nicer solution than my original solution of running the mail forwarder locally. It has also resulted in one more Dockerised application!

I’m intending to convert my other Docker servers over to this approach in the near future. For the system mail part, I still need to create an Ansible role to push out the MSMTP configuration. Actually, the whole question of Ansible/host configuration and how it fits with Dockerised services is still something I need to work out. If anyone has any ideas feel free to share in the comments.

As I said in the last post, I don’t have any more ‘Road to Docker’ posts planned in the immediate future. However, the migration is ongoing so there will be more at some point!

If you liked this post and want to see more, please consider subscribing to the mailing list (below) or the RSS feed. You can also follow me on Twitter. If you want to show your appreciation, feel free to buy me a coffee.

Road to Docker Part 2

My Road To Docker – Part 2: My Home Automation Stack

This post may contain affiliate links. Please see the disclaimer for more information.

This post is part of a series on this project. Here is the series so far:


In my first post of this series, I outlined my plan to convert my infrastructure over to a layered setup. This would consist of virtual machines (in various VLANs), with most of the services running in Docker. This post details the second stage of my road to Docker, although really was is the first stage since I’m writing these out of order! I actually converted my home automation systems over to Docker before tackling the web stack.

The motivation behind upgrading the home automation system first was to do it at the same time as I did a large update to Home Assistant, since I’d been holding back on updating. The main reason for this was the switch to Lovelace as the default UI, which I was dreading. As it turned out, I waited long enough for the awesome HASS developers to make all my problems go away (or at least the Lovelace related ones).

System Summary

I’ve written about my home automation setup before, but here is a brief recap of what I’m running (only the server side stuff):

I had also been running InfluxDB and Grafana. However, something broke in my setup and I hadn’t got around to fixing it. I therefore decided to cut my losses with that and not reinstall it (for now).

Finding Docker Images

Luckily for me, the four main components of my system all have official/recommended Docker images available. This was useful as I’m always pretty reticent to use some questionably maintained image from the Docker Hub, mainly due to the lack of security updates. I also wanted to avoid building custom Docker images for now, until I work out a decent update strategy.

In addition to the four services above I wanted to run the ESPHome dashboard in order to manage my devices better. I had previously just been using the command line tool to build and upload to them. This also has an official Docker image.

Road to Docker Part 2
Looks like I have a few devices to update!

I also ended up running a MaryTTS container to replace PicoTTS that I had been running for my voice announcements. This is due to the lack of PicoTTS inside the HASS Docker image. It was recommended that I use the silversniper/marytts image. Looking at this image, it hasn’t been updated in three years which. This reinforces my point about random images from Docker Hub. Luckily, this isn’t an externally facing application so it isn’t too critical from a security standpoint. However, I think I’ll look into updating that at some point.

Stacking Containers, again…

I set up a new clean VM inside my home automation VLAN. I was a little reserved about doing this, since it means everything in that VLAN (most of which is blocked from the internet) can see the full HA server. However, my main worry over not doing that was the mDNS used by ESPHome. If I can get Avahi/mDNS working across VLANs at some point I will move it. I still have a big network re-organisation to do, so hopefully it will get done then.

The full docker-compose.yml file for my new stack is given below:

There’s nothing particularly earth shattering here. The main point of interest is that I mount the voice file for my preferred MaryTTS voice inside the container. Actually finding the voices is a little interesting. The official way to download them is a GUI tool that won’t run inside the container. I eventually found the XML file which lists all the available voices and extracted the URL of the one I wanted (the online demo helps to decide).

The only other parts worth noting are that I mount all my volumes under /mnt/docker-data, which is an NFS share onto the ZFS array of the virtual machine host. This then gets rolled into my normal backups. I also didn’t bother with a reverse proxy for any of this, since I already have one for HASS sitting in my DMZ (yet to be Dockerised). The other services just get accessed via the machine hostname and port since they are only used internally.

Sometimes Waiting Pays

I didn’t run into any issues particular to setting this up in Docker. At this point I think it’s a pretty well trodden path and this setup is pretty much standard. I did however run into several issues with upgrading to the latest Home Assistant.

First, lets tackle the elephant in the room – Lovelace. I was really worried going into this that it was going to be a huge amount of work. My mind was somewhat put at rest by seeing the UI editor in action via misperry’s video. When I actually came to it, the migration process automatically re-created my existing UI pretty much perfectly. Lesson learned: there is something to be said for waiting for mature software, rather than jumping on the new shiny thing immediately!

Lovelace itself is awesome! The ease of configuration has made me actually focus on making my HASS UI nicer rather than just the bare minimum I could get away with that I had previously. In the screenshot below, you can see my new “Outdoors” panel. This contains weather information, outdoor related sensor readings and a couple of local webcam views.

Road to Docker Part 2
I probably should have taken this screenshot during the day

Remaining Issues

Most of my remaining issues were due to the Home Assistant “Great Migration”. This resulted in a load of entity IDs changing in various components. Obviously, this resulted in my having to update my configuration to change all the names. It took a little while to troubleshoot. This was because if the changed name is used in an automation, the automation has to actually fire to cause an error. In many cases it also just won’t fire, if the name is used in the triggers for the automation.

The final major issue I encountered, was with my frankly awesome vacuuming robot, which appeared to stop reacting to service calls in HASS. The underlying issue appears to be the Botvac D3 returning a different error message than the D7 that the library was tested with. So far this hasn’t been fixed, but I’m currently using the category: 2 workaround suggested and that’s working fine. I think I’ll have a look into fixing that issue and submit a PR when I get time.

Managing Updates

Managing updates to Docker images has always been an bit of an issue for me. In the past I’ve used Watchtower with some success. However, due to the capacity for breaking changes I want to manage HASS updates more carefully. It was suggested to me to just use a bash script which I can run periodically to do this. This isn’t something that had occurred to me before, probably because it’s so simple! Here’s the script I’m using:

This works beautifully and allows me to easily keep up with release to HASS and the other components, once I’ve verified that it’s reasonably safe to update.

Conclusion and Next Steps

Overall I’m pretty happy with how this move has turned out. Once the initial teething issues were all worked out the system has been very stable. I’m appreciating the extra utility of the ESPHome dashboard, which makes it very convenient to update my devices. It’s also great to be back on the latest version of Home Assistant.

In terms of next steps, I would like to give InfluxDB and Grafana another try. My main issue here has always been building the dashboards. It seems to be pretty tricky to get something both good looking and useful in Grafana. I also haven’t seen any pre-built dashboards for use with data from Home Assistant. Perhaps this is because they are so peculiar to individual setups.

I also have an LXD container running ZoneMinder. I’d like to re-deploy this as a Docker container on the same VM. Previously, I’ve not had too much luck running ZoneMinder in Docker. I’ll to see if the situation has improved when I tackle this migration.

I’m not actively working on any further migrations of other services to Docker at the moment, so there will probably be a break in this series for now. However, given my current success I’ll definitely be continuing on with this migration. I just want to work on some other projects for a while!

If you liked this post and want to see more, please consider subscribing to the mailing list (below) or the RSS feed. You can also follow me on Twitter. If you want to show your appreciation, feel free to buy me a coffee.

My Road to Docker: Traefik

My Road to Docker – Part 1: My Web Stack

This post may contain affiliate links. Please see the disclaimer for more information.

This post is part of a series on this project. Here is the series so far:


I’ve been following and using Docker since it’s early days. I think 0.6 was one of the first releases I tried. That said, I’ve never actually had that much luck getting it deployed on meaningful parts of my infrastructure. Usually, I would start some new deployment of something with the intention of using Docker. Then I’d hit some issue which made it difficult (or at least more difficult than it should be). Either that, or I’d run some Docker containers as a testing deployment for a while and then pull them down. It’s safe to say my road to Docker has been long, winding and full of potholes!

Recently, I’ve been giving my infrastructure choices more thought. My main goals are that I want to reduce maintenance and keep the high level of separation and security I have between my different services. As detailed in my last post on my self hosting setup, I’ve been using LXD fairly successfully. The problem with this is really that LXD containers are more like virtual machines. Each container is a little Linux system which needs to be kept up to date, monitored and generally looked after. When you have one container per service this becomes a chore. In actuality I don’t have one LXD container per service. Instead I group dependent services such as databases with their parent. However, I’ve still ended up with quite a few containers.

Decisions, decisions…

I really like the separation I’m able to have by keeping my LXD containers on separate VLANs. Docker does support this via macvlan, but the last time I played with it I had to manually assign IPs to each container. It looks like Docker will now do this automatically. However, it allocates from it’s own IP pool and won’t use your DHCP server. This means I can’t assign IPs from my Pfsense firewall. Also having one public (LAN) IP per Docker service kinda sucks and I also don’t want to rely on just one project for the security of my system.

Additionally, I have the need to run at least one VM anyway for my virtualised Pfsense firewall. This brings me to the idea of running Docker inside VMs. Each VM can be assigned to the relevant VLAN interface and get it’s IP from DHCP. I also get the benefit of two levels of isolation using different technologies. I haven’t yet decided on whether to use Proxmox or Ubuntu Server+Libvirt+Cockpit for the host systems. Hopefully this will be the subject of a future post.

On To Today

All the above is pretty much background, because I actually don’t want to talk about my locally hosted stuff today. Instead I’m going to talk about my stuff in the cloud – specifically the hosting for this site.

I’ve used Linode since around 2011 when I set up my self hosted mail server. Since this time, I’ve hosted this site on a fairly standard LAMP setup on that same server. The mail server install is now getting on a bit, but being CentOS based it’s all still supported for updates until next year. However, in preparation for re-building the mail server I decided to move the web stuff out onto another server and run it in Docker. This basically gives me the same setup of Docker-on-VM as I have on my local infrastructure. Just the hypervisor UI differs – it’s all KVM underneath.

Since I was moving the site to a new server, I also decided to move closer to my audience. According to my Matomo statistics this is predominantly US based. To this end I span up a new $5 Linode (Nanode), running Ubuntu 18.04 in their New Jersey datacenter. This should also be pretty fast for European visitors (to be honest it’s lightning fast even from NZ).

Getting Going

After doing some pretty standard first 10 minutes on a server stuff, I set about installing Docker. Instead of installing via my usual method of adding the APT repository I decided to see how installing from a Snap would work in production:

A few seconds later Docker was installed. However, I ran into a wrinkle when trying to add myself to the Docker group. Basically, it wouldn’t allow me to run Docker commands without root access (still). The solution is to add the Docker group before installing the Snap, so I removed it, added the group and reinstalled. So full instructions for installing Docker via Snap should more accurately be:

Stacking Containers

I’d resolved to install the blog inside the official WordPress container and connect use mariadb for the database. Since I was moving my install over, it was slightly more complex than it would be for a clean install, but this article put me on the right track.

The main issue I encountered didn’t happen until I had the site running and was trying to get HTTPS running. Due to Let’s Encrypt’s HTTP verification this had to be done after the DNS settings were updated i.e. the site was live. This issue manifested as WordPress serving HTTP URLs for all links and embedded content. This leads to mixed content warnings/blocks in Firefox and Chrome. The issue seems to be common to any setup where HTTPS is handled by a reverse proxy. Basically, in this setup WordPress isn’t aware that it should be using HTTPS and needs to be told. Adding the following to wp-config.php fixes the problem:

I decided to use Traefik as my reverse proxy, which I’m quite pleased with. The automated service discovery is pretty awesome and will really come into it’s own on my other self hosted infrastructure. I also whacked a Varnish cache (from this Docker image) in between. So far I haven’t done much with Varnish, but it’s there for when I get time to tweak it.

Moving Matomo

I also moved my Matomo Analytics instance to the new server using the official Docker image. This gave me much less trouble than WordPress since I just allowed it to use the embedded Matomo version with my configuration file and database from the old install.

I connected Matomo up directly to my Traefik instance, without running it through Varnish, to avoid any potential issues with the cache interfering with my statistics. Traefik just worked with this setup, which was pretty refreshing.

My Road to Docker: Traefik
My Traefik Dashboard

PSA: Docker will Ignore Your Firewall!

Along the way, I ran into another issue. Before I changed the DNS settings to make the site live, I was binding Traefik to port 8080 and accessing it via an SSH tunnel for making changes in WordPress. Since I had configured UFW to block everything (except SSH) when I set up the server, I thought this was a nice secure setup for debugging.

I was wrong.

I only noticed something was off because I happened to have the log output from Traefik open and saw random IPs in the logs. Luckily, the only virtual host I had configured at that point was localhost:8080 so all the unwanted visitors got 404 responses. Needless to say I pulled down all the containers until I could work out what was going on.

This appears to be a known interaction between Docker and any firewall utility (including both UFW and Firewalld). The issue is inherent in the way that Docker uses iptables to route traffic to your containers. Basically, the port forwarding rules go in the NAT chain in iptables. This means incoming traffic is re-routed before it hits the INPUT chain containing the rules from UFW/Firewalld.

I tried the fix suggested – disabling iptables support in Docker, but this completely broke inter-container connectivity (unsurprising, when you think about it). My solution for now is just to be really careful when binding ports. Make sure that any ports you don’t want to give access to the outside world are bound only to 127.0.0.1. There is also the DOCKER-USER iptables chain if you need more flexibility, but it means you need to use raw iptables rules.

This issue is a major security flaw and needs to be given more attention. Breaking administrator expectations of security like this is going to lead to loads of services being exposed to the big bad Internet that really shouldn’t be!

My Full Web Stack

Below is the docker-compose.yml file for my full web stack. I store any secret variables in an env.sh file which I source before running my docker-compose commands.

This is all pretty much as described, however there are two notable points:

  • I went for separate mariadb containers for the databases for WordPress and Matomo. This is because the official mariadb container only supports creation of one database/user via environment variables. I’m not super happy with this arrangement, but it is working well and doesn’t seem to use too much memory.
  • The aliases portion under the network configuration for the Traefik container allow the other containers to route internal requests back to themselves. This helps with things such as the loopback check in WordPress, which will otherwise fail.

That’s pretty much all there is to it!

Conclusion

Overall, I’m really happy with the way this migration has worked out. I’ve even been able to downgrade the Linode plan for the original server to the $5/month plan, since it now has less load. This means I’m paying the same as I was for the single server for two servers, although with half the RAM each. I think I paid an extra $2-3 during the migration period, since that took me a while to compete. Even that isn’t too bad.

I’ve already started on further Docker migrations on some of the infrastructure I have on my home servers. These should be the subject of further posts.

If you liked this post and want to see more, please consider subscribing to the mailing list (below) or the RSS feed. You can also follow me on Twitter. If you want to show your appreciation, feel free to buy me a coffee.