docker build gitlab

My Road to Docker – Part 4: Automated Container Builds with GitLab CI

This post may contain affiliate links. Please see the disclaimer for more information.

This post is part of a series on this project. Here is the series so far:


In the course of migrating most of the services I run over to running as Docker containers I’ve mostly tried to steer clear of building my own images. Thanks to the popularity of Docker this has been pretty easy for the most part. Many projects now ship an official Docker image as part of their releases. However, the inevitable happened when I had to build my own image in my recent TT-RSS migration.

The reason for my reluctance to build my own images is not some unfounded fear of Dockerfiles. It’s not building the images that I’m worried about, it’s maintaining them over time.

Every Docker image contains a mini Linux distribution, frozen in time from the day the image was built. Like any distribution of Linux these contain many software packages, some of which are going to need an update sooner or later in order to maintain their security. Of course you could just run the equivalent of an apt upgrade inside each container, but this defeats some of the immutability and portability guarantees which Docker provides.

The solution to this, is basically no more sophisticated than ‘rebuild your images and re-deploy’. The problem is no more help is offered. How do you go about doing that in a consistent and automated way?

Enter GitLab CI

If you have been following this blog you’ll know I’m a big fan of GitLab CI for these kinds of jobs. Luckily, GitLab already has support for building Docker images. The first thing I had to do was get this set up on my self-hosted GitLab runner. This was complicated by the fact that my runner was already running as a Docker container. It’s basically trying to start a Docker-in-Docker container on the host, from inside a Docker container, which is a little tricky (if you want to go further down, Docker itself is running from a Snap on a Virtual Machine! It really is turtles all the way down).

After lots of messing around and tweaking, I finally came up with a runner configuration which allowed the DinD service to start and the Docker client in the CI build to see it:

concurrent = 4
check_interval = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "MyRunner"
  url = "https://gitlab.com/"
  token = "insert your token"
  executor = "docker"
  [runners.custom_build_dir]
  [runners.docker]
    tls_verify = false
    image = "ubuntu:18.04"
    privileged = true
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    environment = ["DOCKER_DRIVER=overlay2", "DOCKER_TLS_CERTDIR=/certs"]
    volumes = ["/certs/client", "/cache"]
    shm_size = 0
    dns = ["my internal DNS server"]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
  [runners.custom]
    run_exec = ""

The important parts here seem to be that the privileged flag must be true in order to allow the starting of privileged containers and that the DOCKER_TLS_CERTDIR should be set to a known location. The client subdirectory of this should then be shared as a volume in order to allow clients to connect via TLS.

You can ignore any setting of the DOCKER_HOST environment variable (unless you are using Kubernetes, maybe?). It should be populated with the correct value automatically. If you set it you will most likely get it wrong and the Docker client won’t be able to connect.

The CI Pipeline

I started the pipeline with a couple of static analysis jobs. First up is my old favourite yamllint. The configuration for this is pretty much the same as previously described. Here it’s only really checking the .gitlab-ci.yml file itself.

Next up is a new one for me. hadolint is a linting tool for Dockerfiles. It seems to make sensible suggestions, though I disabled a couple of them due to personal preference or technical issues. Conveniently, there is a pre-built Docker image, which makes configuring this pretty simple:

hadolint:
  <<: *preflight
  image: hadolint/hadolint
  before_script:
    - hadolint --version
  script:
    - hadolint Dockerfile

Building the Image

Next comes the actual meat of the pipeline, building the image and pushing it to our container registry:

variables:
  IMAGE_TAG: registry.gitlab.com/robconnolly/docker-ttrss:latest

build:
  stage: build
  image: docker:latest
  services:
    - docker:dind
  before_script:
    - docker info
  script:
    - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com
    - docker build -t $IMAGE_TAG .
    - docker push $IMAGE_TAG
  tags:
    - docker

This is relatively straightforward if you are already familiar with the Docker build process. There are a couple of GitLab specific parts such as starting the docker:dind service, so that we have a usable Docker daemon in the build environment. The login to the GitLab registry uses the built-in $CI_BUILD_TOKEN variable as the password and from there we just build and push using the $IMAGE_TAG variable to name the image. Simple!

docker build gitlab
The pipeline for this is actually rather simple and boring compared to some of my others!

I also added another pipeline stage to send me a notification via my Gotify server when the build is done. This pretty much just follows the Gotify docs to send a message via curl:

notify:
  stage: notify
  image: alpine:latest
  before_script:
    - apk add curl
  script:
    - |
      curl -X POST "https://$GOTIFY_HOST/message?token=$GOTIFY_TOKEN" \
           -F "title=CI Build Complete" \
           -F "message=Scheduled CI build for TT-RSS is complete."
  tags:
    - docker

Of course the $GOTIFY_HOST and $GOTIFY_TOKEN variables are defined as secrets in the project configuration.

You can find the full .gitlab-ci.yml file for this pipeline in the docker-ttrss project repository.

Scheduling Periodic Builds

GitLab actually has support for scheduled jobs built in. However, if you are running on gitlab.com as I am you will find that you have very little control over when your jobs actually run, since it runs all cron jobs at 19 minutes past the hour. This would be fine for this single job, but I’m planning on having more scheduled builds run in the future, so I’d like to distribute them in time a little better, in order to reduce the load on my runner.

docker build gitlab
Pipeline triggers are created via the Settings->CI/CD menu in GitLab.

In order to do this I’m using a pipeline trigger, kicked off by a cron job at my end:

# Run TT-RSS build on Tuesday and Friday mornings
16  10 *   *   2,5   /usr/bin/curl -s -X POST -F token=SECRET_TOKEN -F ref=master https://gitlab.com/api/v4/projects/2707270/trigger/pipeline > /dev/null

This will start my pipeline at 10:16am (NZ time) on Tuesdays and Fridays. If the build is successful there will be a new image published shortly after these times.

An Aside: Managing random cron jobs like these really sucks once you have more than one or two machines. More than once I’ve lost a job because I forgot which machine I put it on! I know Ansible can manage cron jobs, but you are still going to end up with them configured across multiple files. I also haven’t found any way of managing cron jobs which relate to a container deployment (e.g. running WordPress cron for a containerised instance). Currently I’m managing all this manually, but I can’t help feeling that there should be a better way. Please suggest any approaches you may have in the comments!

Updating Deployed Containers

I could extend the pipeline above to update the container on my server with the new image. However, this would leave the other containers on the server without automated updates. Therefore I’m using Watchtower to periodically check for updates and upgrade any out of date images. To do this I added the following to the docker-compose.yml file on that server:

watchtower:
    image: containrrr/watchtower
    environment:
      WATCHTOWER_SCHEDULE: 0 26,56 10,22 * * *
      WATCHTOWER_CLEANUP: "true"
      WATCHTOWER_NOTIFICATIONS: gotify
      WATCHTOWER_NOTIFICATION_GOTIFY_URL: ${WATCHTOWER_NOTIFICATION_GOTIFY_URL}
      WATCHTOWER_NOTIFICATION_GOTIFY_TOKEN: ${WATCHTOWER_NOTIFICATION_GOTIFY_TOKEN}
      TZ: Pacific/Auckland
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    restart: always

Here I’m just checking for container updates four times a day. However, these are divided into two pairs of sequential checks separated by 30 minutes. This is more than sufficient for this server since it’s only accessible from my local networks. One of the update times is scheduled to start ten minutes after my CI build to pick up the new TT-RSS container. The reason for the second check after 30 mins is to ensure that the container gets updated even if the CI pipeline is delayed (as sometimes happens on gitlab.com).

You’ll note that I’m also sending notifications from Watchtower via Gotify. I was pleasantly surprised to discover the Gotify support in Watchtower when I deployed this. However, I think the notifications will get old pretty quickly and I’ll probably disable them eventually.

Conclusion

With all this in place I no longer have to worry about keeping my custom Docker images up to date. They will just automatically build at regular intervals and be updated automatically on the server. It actually feels pretty magical watching this all work end-to-end since there are a lot of moving parts here.

I’ll be deploying this approach as I build more custom images. I also still have to deploy Watchtower on all my other Docker hosts. The good news is that this has cured my reluctance to build custom images, so there will hopefully be more to follow. In the meantime, the community can benefit from a well maintained and updated TT-RSS image. Just pull with the following command:

docker pull registry.gitlab.com/robconnolly/docker-ttrss:latest

Enjoy!

If you liked this post and want to see more, please consider subscribing to the mailing list (below) or the RSS feed. You can also follow me on Twitter. If you want to show your appreciation, feel free to buy me a coffee.

shed lights rainbow

Smart Outdoor Solar Powered Lighting with the ESP32

This post may contain affiliate links. Please see the disclaimer for more information.

The primary form of heating in our home is a wood burning stove. You’re probably thinking that this isn’t very ‘smart’ and you’d be right. It’s the least smart part of my smarthome, requiring daily lighting and monitoring during the winter months. What it is though is eco-friendy and cheap to run. Also, nothing quite beats a wood fire for warmth and hygge. The point of all this is that we have a wood shed (two actually) at the back of our house. During dark winter nights, trips to these have been fraught with danger and mystery due to a lack of lighting. That is until now: this winter my completely over-engineered, ESP32 driven, solar powered, smart LED lights will light the way!

Of course, I could just use a head torch for my after dark trips to the wood shed. Or buy any number of (doubtless cheap and nasty) solar powered lights from the local DIY store. However, that didn’t seem very fun and wouldn’t integrate with Home Assistant and the rest of my smarthome.

Components

For these lights I decided to use WS2812B light strip driven by an ESP32. The solar power system would utilise an off the shelf solar controller and lead acid battery along with a 5W solar panel which I acquired cheap from work.

Full Parts List

solar powered esp32 panel specs
The specs of the solar panel

GCode for the Ender 3 is also available for the 3D printed parts. If you have another printer you’ll need to slice the STL files yourself. I had a few issues with some of the small brackets adhering to the bed, but I was able to get a high enough yield thanks to the OctoPrint-Cancelobject plugin.

led brackets
The LED brackets
battery bracket
The battery bracket

The charge controller is rather over-spec’d for the LEDs I’m actually running (which pull less than 2A at the 12V battery voltage, on full brightness). This is because I originally thought I would use the full 5m of LED strip that I ordered. However when it arrived and I saw how bright they are I downgraded to just 2m. I had also ordered a beefier power supply for this purpose. The one I ended up using was left over from my failed tablet salvaging efforts. This means I have 3m of LED strip and a decent power supply left over for a future project.

Putting it Together

I spent the better part of a day putting together the components and soldering all the cabling. I then spent at least another day mounting things on the shed and fixing issues. All the joins between cables are double covered with heatshrink tubing. The first layer insulates the cable from any neighbouring cables. The second layer is for waterproofing and has roof sealant injected into it. The same is done on the joins between the cable and the LED strip. Hopefully this will stand up to the intense New Zealand rain. Seriously, you have not seen rain until you’ve seen NZ rain! Luckily most of the components are mounted out of the worst of the weather. The cables enter the box through the IP68 cable gland which should be weather tight.

solar powered esp32 mounting
Mounting the electronics in the case
mounting battery
The battery is held securely with that bracket
led mounting bracket
Mounting the first LED bracket
led strip mounted
The LED strip mounted to the shed

When I tested the LEDs after soldering the lead out wires to them I found that the colours were off and transitions and effects were flickery. This could have been due to either a power issue or an issue on the data line. Measuring the 5V line showed minimal voltage drop there. Since the LEDs were OK before adding the lead out I went with the data issue.

solar powered esp32 sacrificial pixel
The sacrificial pixel assembly before mounting in the case

The issue turned out to be due to voltage drop on the 3.3V signal from the ESP32 along the lead out cable. The data line is quite sensitive to voltage drop here because the LEDs are supposed to receive a 5V data signal. They work with 3.3V, but not much lower. In order to solve this I added an single extra pixel at the microcontroller end to boost the signal voltage from 3.3V to 5V. This solves the issue since the data signal is being amplified by each pixel in the chain.

ESPHome Code

The ESPHome code to drive the lights is fairly simple. First we start with the standard setup:

---
esphome:
  name: shed_lights
  platform: ESP32
  board: esp32doit-devkit-v1

wifi:
  ssid: !secret wifi_ssid2
  password: !secret wifi_passwd2
  use_address: !secret shed_lights_ip

mqtt:
  broker: !secret mqtt_broker
  username: !secret mqtt_user
  password: !secret mqtt_passwd

# Enable logging
logger:

ota:
  password: !secret ota_passwd

Here we define the board type, set up the wifi and MQTT connections, enable logging and set up OTA updates. If you’re wondering why I’m using MQTT rather than the ESPHome API, it’s for no other reason than I like MQTT!

power_supply:
  - id: 'led_power'
    pin:
      number: GPIO25
      inverted: true

Next I set up a power supply component. This is used along with the single channel relay to automatically power up and down the power supply to the LEDs. This will save a bit of power and also makes sure that there is no power flowing in the cables outside the box for most of the time, which may help in the event of a leaky connection. In order to do this I’m not running the ESP32 from the same 5V supply, instead using one of the USB ports from the solar controller.

The LED strip configuration is then pretty standard:

light:
  - platform: fastled_clockless
    chipset: WS2812B
    pin: GPIO23
    num_leds: 61
    rgb_order: GRB
    name: "Shed Lights"
    effects:
      - addressable_rainbow:
      - addressable_color_wipe:
      - addressable_scan:
      - addressable_twinkle:
      - addressable_random_twinkle:
      - addressable_fireworks:
      - addressable_flicker:
    power_supply: 'led_power'

Note that there are 61 pixels here, that 2m at 30 pixels per meter plus one sacrificial voltage boosting pixel. The addition of the effects is a bit of a gimmick since I’m mostly interested in white light for the application. I only bought the RGB LEDs because the price difference wasn’t enough to justify only buying white.

solar powered esp32 finished
The finished electronics mounted to the shed
solar powered esp32 panel
The solar panel is mounted to the fence behind the shed
solar powered esp32 relative positions
The electronics box and panel on the end of the shed

Voltage Sensing and Health Monitoring

After putting all this together and mounting it on the shed I decided that I’d like to have some form of monitoring for the voltages from the battery and solar panel. The solar controller obviously monitors these but there is no way to get this data out.

In the end I soldered up a couple of voltage divider circuits and added these to the setup in the box. I also added a DHT22 sensor for temperature and humidity sensing inside the box.

voltage monitoring schematic
Schematic of the voltage dividers used for the power monitoring circuits

The ESPHome configuration for these follows. I also added a binary status sensor and a WiFi signal sensor to allow me to monitor the system remotely.

binary_sensor:
  - platform: status
    name: "Shed Lights Status"

sensor:
  - platform: wifi_signal
    name: "Shed WiFi Signal"
    update_interval: 180s
  - platform: dht
    pin: GPIO15
    model: AM2302
    temperature:
      name: "Shed Battery Box Temperature"
    humidity:
      name: "Shed Battery Box Humidity"
    update_interval: 180s
  - platform: adc
    pin: GPIO39
    name: "Shed Battery Voltage"
    icon: "mdi:car-battery"
    attenuation: "11db"
    filters:
      - multiply: 4.24
      - sliding_window_moving_average:
          window_size: 12
          send_every: 12
    update_interval: 15s
  - platform: adc
    pin: GPIO36
    name: "Shed Solar Panel Voltage"
    icon: "mdi:solar-panel"
    attenuation: "11db"
    filters:
      - multiply: 5.73
      - filter_out: 0.00
      - sliding_window_moving_average:
          window_size: 12
          send_every: 12
    update_interval: 15s

The voltage readings I am getting from the two voltage sensors are a little weird. The battery voltage is higher than I would expect and the solar panel voltage is lower. I double checked the multiplication factors against the raw ADC readings and the resistors used and the readings make sense. Initially I thought this could be due to the temperature inside the box (~50°C when it’s in full sun!), but now I’m not so sure since this is also the case at lower temperatures. It could be the behaviour of the charge controller. I’ll continue to monitor it over different charge states.

shed lights hass card
The card in Home Assistant showing the system status and lighting controls

The Finished Product

Now the moment you’ve all been waiting for – gratuitous photos of the LEDs in fancy colours!

shed lights white
In white, the intended operation mode
shed lights red
Red is insanely bright!
shed lights green
Green
shed lights blue
Blue
shed lights purple
Purple/pink, the favourite of some people in the household
shed lights rainbow
Finally the rainbow effect, the other effects don’t come out so well in a still photo

Conclusion

This is actually my first time using the WS2812B LED strip and I have so say I’m really impressed. You can be sure there will be other LED lighting projects coming in future now that I’ve dipped my toes in!

I’m really pleased with the final product. The LEDs look awesome and provide ample light for their task. The ESPHome base has so far been rock solid in terms of stability, which is what I’ve come to expect from using it in other projects.

The initial intention was to make these lights motion activated. However, I couldn’t find a motion sensor which was suitable for outdoor use. I’d also have to locate the sensor at the other end of the sheds from the battery box which would mean a whole load of wiring. As such I’ve decided to build by own wireless outdoor motion sensor and connect it to my MySensors network. I’ll then trigger the LEDs via an automation in Home Assistant. I’ll post an update on this when I have it running.

This has been a really fun and interesting project. As always, please let me know what you think in the feedback channels and feel free to share your own LED lighting projects.

If you liked this post and want to see more, please consider subscribing to the mailing list (below) or the RSS feed. You can also follow me on Twitter. If you want to show your appreciation, feel free to buy me a coffee.

mysensors node red proxy flow

Quick Project: MySensors MQTT Proxy with Node-RED

This post may contain affiliate links. Please see the disclaimer for more information.

In my previous exploration of MySensors, I encountered issues with the range of the nrf24l01+ radio. One of the simplest solutions to this problem is to move the gateway closer to the intended deployment of the sensors. In my case the sensor is positioned on the back wall of our house, just outside the master bedroom. Connected to the TV inside the master bedroom is a Raspberry Pi running Libreelec. I reasoned that I should be able to use this Pi to proxy the MySensors serial data to Home Assistant via MQTT, using the MySensors Node-RED nodes. With this approach I can make a new MQTT gateway using hardware I already have deployed.

It was also suggested that adding a 10-100uF 6.3/10V capacitor across the power line to the radio would also help with the range issues. I’m definitely going to do this, but the components haven’t arrived yet!

Hardware Setup

If you followed my previous post all you need to do here is plug in the serial gateway to the Pi! Otherwise you’re going to need to build a gateway (and maybe some sensors). Go check out my original post for more details.

It’s probably worth going through how this is going to work. Basically, the idea is that the RF data will be received by our existing MySensors gateway and sent via serial to the RPi. This will be read by Node-RED, re-encoded and sent via MQTT to Home Assistant which will still act as the MySensors controller:

Serial GW ⇒ Serial Connection ⇒ Node-RED (on RPi) ⇒ MQTT ⇒ HASS (Controller)

In the return direction this dataflow operates in reverse:

HASS ⇒ MQTT ⇒ Node-RED ⇒ Serial Connection ⇒ Serial GW

Deploying Node-RED on Libreelec

Libreelec is a cut down appliance-like Linux distribution, specifically for running Kodi. At first glance it seems like installing a third party program such as Node-RED would be difficult. However, Libreelec has Docker support via an add-on, which makes things as easy as running the command:

$ docker run -d --restart=always --device=/dev/ttyUSB0:/dev/ttyUSB0 --group-add dialout -v /storage/node-red-data:/data -p 1880:1880 --name node-red nodered/node-red:latest

This starts the Node-RED Docker container with access to the serial port for our plugged in Arduino board. You’ll need to adjust the path to the TTY device depending on how the Arduino gets enumerated. We are also storing the data for Node-RED (including flows) in the Libreelec storage directory, so that we can re-create the container without losing our flows.

Once this starts up, you should be able to access a nice clean Node-RED instance on port 1880 of your Libreelec Pi. At this point it’s probably a good idea secure your instance and set up projects for backing up your flows via Git.

MySensors Proxy Flow

So here’s what you came here for. Using the MySensors nodes I’ve created a flow which receives MySensors data via the serial port decodes it to a Javascript object and then immediately encodes it and sends it via MQTT. To do this we make use of the serial decode and MQTT encode nodes. In the opposite direction we receive MQTT data on the mysensors/in/# wildcard and decode it to another JS object. This is then immediately encoded for serial and sent out the serial port. The finished flow looks like this:

mysensors node red proxy flow
The MySensors proxy flow

The full JSON for this is below. You’ll need to update the details for the MQTT broker and serial port to match your installation:

[{"id":"750924cf.e87e14","type":"tab","label":"MySensors MQTT Proxy","disabled":false,"info":""},{"id":"e7a50a55.c700d8","type":"mysdecode","z":"750924cf.e87e14","database":"","name":"Decode Serial","mqtt":false,"enrich":false,"x":244,"y":125,"wires":[["1fe04ed1.423d09"]]},{"id":"1fe04ed1.423d09","type":"mysencode","z":"750924cf.e87e14","name":"Encode MQTT","mqtt":true,"mqtttopic":"mysensors/out","x":429,"y":125,"wires":[["73eb8bcf.3be6bc"]]},{"id":"73eb8bcf.3be6bc","type":"mqtt out","z":"750924cf.e87e14","name":"","topic":"","qos":"","retain":"","broker":"61f77eeb.b75e2","x":584,"y":125,"wires":[]},{"id":"1a5ca295.7db125","type":"serial in","z":"750924cf.e87e14","name":"serial in","serial":"31aea0d6.e5a438","x":87,"y":125,"wires":[["e7a50a55.c700d8"]]},{"id":"5a8336a4.b9d398","type":"mqtt in","z":"750924cf.e87e14","name":"","topic":"mysensors/in/#","qos":"2","datatype":"auto","broker":"61f77eeb.b75e2","x":117,"y":179,"wires":[["b8c6fd6f.3ecf08"]]},{"id":"b8c6fd6f.3ecf08","type":"mysdecode","z":"750924cf.e87e14","database":"","name":"Decode MQTT","mqtt":true,"enrich":false,"x":320,"y":180,"wires":[["440bfad4.44eadc"]]},{"id":"440bfad4.44eadc","type":"mysencode","z":"750924cf.e87e14","name":"Encode Serial","mqtt":false,"mqtttopic":"","x":508,"y":179,"wires":[["e8c5f4bd.bc3d3"]]},{"id":"e8c5f4bd.bc3d3","type":"serial out","z":"750924cf.e87e14","name":"serial out","serial":"31aea0d6.e5a438","x":676,"y":179,"wires":[]},{"id":"61f77eeb.b75e2","type":"mqtt-broker","z":"","name":"Home Broker","broker":"mqtt.example.com","port":"1883","tls":"b8f1023d.4df4","clientid":"","usetls":false,"compatmode":true,"keepalive":"60","cleansession":true,"birthTopic":"nodered/status","birthQos":"2","birthRetain":"true","birthPayload":"online","closeTopic":"nodered/status","closeQos":"2","closeRetain":"true","closePayload":"offline","willTopic":"nodered/status","willQos":"2","willRetain":"true","willPayload":"offline"},{"id":"31aea0d6.e5a438","type":"serial-port","z":"","serialport":"/dev/ttyUSB0","serialbaud":"115200","databits":"8","parity":"none","stopbits":"1","waitfor":"","dtr":"none","rts":"none","cts":"none","dsr":"none","newline":"\\n","bin":"false","out":"char","addchar":"","responsetimeout":"10000"},{"id":"b8f1023d.4df4","type":"tls-config","z":"","name":"Home Broker","cert":"","key":"","ca":"","certname":"","keyname":"","caname":"","verifyservercert":true}]

That’s it for the flow. We can integrate this into HASS by changing our previous gateway configuration to use MQTT:

mysensors:
  gateways:
    - device: mqtt
      persistence_file: '/config/mysensors.json'
      topic_in_prefix: 'mysensors/out'
      topic_out_prefix: 'mysensors/in'

Conclusion

With this in place I’m now able to receive data from my sensor in the greenhouse without the reception issues I was experiencing. I like the use of Node-RED for these kinds of protocol conversion jobs. Its dataflow model is supremely suited to these operations.

Other than the range issue, the MySensors node I built in my previous post has been pretty reliable. I did make a minor hardware change to the setup I described earlier, which was to move positive battery connection from the raw power line to the VCC line of the Arduino. This is because the raw line is regulated to 3.3V. The voltage regulator will consume some power and also won’t work once the battery voltage drops below about 3.3V. Connecting to VCC powers the micro directly from the battery without the extra power loss. Since this modification the sensor has been running flawlessly for several weeks.

If you liked this post and want to see more, please consider subscribing to the mailing list (below) or the RSS feed. You can also follow me on Twitter. If you want to show your appreciation, feel free to buy me a coffee.

home assistant wall mounted tablet

Home Assistant Wall Mounted Tablet Update

This post may contain affiliate links. Please see the disclaimer for more information.

This is an update to my previous post on my wall mounted Home Assistant tablet. There have been quite a few changes, so I thought it was worth another post on the subject. Let’s get into it…

Power Failure

I’ll start with the big issue which has plagued this project – power. Since the battery in the original tablet was completely exhausted I had built a wired power supply to replace it. Unfortunately, this wasn’t powerful enough and the tablet would power off sometimes under high load. This problem seemed to get worse until the tablet would power off after just a few minutes, thus becoming pretty much useless.

home assistant wall mounted tablet
The new PSU all wired up (difficult to capture in a photo!)

I mentioned previously that I had ordered a 5A power supply to fix this problem, but even that didn’t work. The tablet must pull transient currents of more than 5A, which a battery can supply quite easily. Unfortunately the fixed power supply cannot. This is a real shame because I put a lot of effort into the new power supply – I even printed a case for it based on a modified version of the parametric PCB case I used in my recent MySensors build.

Even with the failure of the new power supply, the exercise in building this has demonstrated the utility of a wall mounted tablet in our household. The decision was therefore taken to replace the old broken tablet with a new one.

The New Tablet

The new tablet is the Lenovo Tab E8 (link is to Amazon, but ours came from a local retailer), chosen solely because it was the cheapest 8 inch tablet available!

home assistant wall mounted tablet
The new tablet mounted on the wall, complete with new brackets

The tablet seems pretty good overall, at least for the price. The build quality is fine, although the case is very plastic-y. It’s a little slow after boot up as it loads up everything, but in normal usage as a wall mounted dashboard it performs perfectly well. The shipped operating system is Android 7.0, which is up to date enough to not present any problems with app support.

The biggest issue with this device is that it charges really slowly, even from the provided charger. I even had cases when setting it up that the battery would still drain when plugged in to the charger. I was a little concerned that the tablet battery would become discharged over time during normal use, due to wake ups from the motion sensing. However, once the tablet was mounted on the wall and left overnight it charged to 100% and has remained there since.

Fixing the Brackets

I printed new brackets for the tablet with a reduced front lip. This removes the issue with the brackets obscuring the screen. The right bracket (left from the point of view of the tablet and in the STL files) also has a cut out in it’s side to allow the USB cable to pass through. I made these edits in OpenSCAD, by first importing the original STL files. I then created an object which represents the material I wanted to remove. Then a difference operation between the two gives the final object. I’m pretty pleased with them. The only thing I wasn’t able to reproduce was the rounded internal edges on the cut out sections.

Wall Panel MQTT Integration

With the power issues solved, I can integrate the tablet fully with Home Assistant (without fear that the extra load will cause it to power off). I’ve integrated in the sensors available on this device (pretty much just the power status). The full YAML code for this is available on GitLab. This pretty much went according to the WallPanel documentation. However, I did find that I needed to specify a value_template in order to extract the battery value:

sensor:
  - platform: mqtt
    state_topic: "wallpanel/kitchen/sensor/battery"
    name: "Kitchen WallPanel Battery Level"
    unit_of_measurement: "%"
    value_template: '{{value_json.value}}'
    device_class: battery

To extract the boolean status for the charging state sensors, the template gets a little more complex:

binary_sensor:
  - platform: mqtt
    state_topic: "wallpanel/kitchen/sensor/battery"
    name: "Kitchen WallPanel Charge State"
    value_template: '{% if value_json.charging %}ON{% else %}OFF{% endif %}'
    device_class: power

  - platform: mqtt
    state_topic: "wallpanel/kitchen/sensor/battery"
    name: "Kitchen WallPanel AC State"
    value_template: '{% if value_json.acPlugged %}ON{% else %}OFF{% endif %}'
    device_class: plug

I also integrated in the motion, face detection and QR code sensors, although I’m not using the face detection or QR code sensors and have since disabled them in WallPanel.

sensor:
  - platform: mqtt
    state_topic: "wallpanel/kitchen/sensor/qrcode"
    name: "Kitchen QR Code"
    value_template: '{{value_json.value}}'

binary_sensor:
  - platform: mqtt
    state_topic: "wallpanel/kitchen/sensor/motion"
    name: "Kitchen Motion"
    payload_on: '{"value":true}'
    payload_off: '{"value":false}'
    device_class: motion

  - platform: mqtt
    state_topic: "wallpanel/kitchen/sensor/face"
    name: "Kitchen Face Detected"
    payload_on: '{"value":true}'
    payload_off: '{"value":false}'
    device_class: motion

The final integration I did was to add a switch to turn on and off the camera streaming. However, in general usage this seems a bit flakey, with the camera stream not usually becoming available until after a WallPanel restart. Here’s the YAML used for the switch anyway:

switch:
  - platform: mqtt
    command_topic: "wallpanel/kitchen/command"
    name: "Kitchen Camera State"
    payload_on: '{"camera": true}'
    payload_off: '{"camera": false}'
    optimistic: true
    retain: true

Dashboard Improvements

home assistant wall mounted tablet
My Current Dashboard

I’ve improved my dashboard somewhat since my last post. However, it’s still a long way from some of the awesome dashboards I’ve seen posted. It’s still a work in progress!

The main changes have been to switch to the animated weather custom card for the weather forecast and to add Custom Header to my install to compact the header. I then added more buttons, some media controls and a gauge showing the progress of my 3D prints. The vacuum buttons call some custom scripts to start the vacuum with our preferred settings. This is necessary to work around a bug in HASS with the Botvac D3:

script:
  spot_clean:
    alias: "Spot Clean"
    sequence:
      - alias: "Start spot clean"
        service: vacuum.neato_custom_cleaning
        data:
          entity_id: vacuum.marvin
          mode: 2
          category: 3

  house_clean:
    alias: "House Clean"
    sequence:
      - alias: "Start house clean"
        service: vacuum.neato_custom_cleaning
        data:
          entity_id: vacuum.marvin
          mode: 1
          category: 2

It should be noted that I’m running a slightly older version of HASS at the moment. In newer versions the service name changed to neato.custom_cleaning. Also, using a category value of 3 for spot cleaning appears to be undocumented, but if you look at the library, that’s what it’s doing.

Conclusions

This setup is now working perfectly and all the problems with the previous iteration have been resolved. It’s unfortunate that this was done by throwing money at the problems, but sometimes things just don’t work out as we hope!

Overall, I’m really happy with the finished product. The dashboard will be continually developing as I come up with ideas and work out what’s useful and what isn’t. Feel free to share your ideas again via the feedback channels.

If you liked this post and want to see more, please consider subscribing to the mailing list (below) or the RSS feed. You can also follow me on Twitter. If you want to show your appreciation, feel free to buy me a coffee.

tiny tiny rss docker

Self-Hosted RSS with Tiny Tiny RSS in Docker

This post may contain affiliate links. Please see the disclaimer for more information.

With the rise of social media, RSS seems to have been largely forgotten. However, there are still those who are dedicated enough to keep curating their own list of feeds and plenty of software to support them. I’ve always been a fan of RSS and believe it’s probably time for a resurgence in use that would free us from our algorithmic overlords. As such I’ve run an instance of Tiny Tiny RSS for several years and the time has come to migrate it to Docker.

It’s relatively unknown that you can still get RSS feeds for most news sources on the Internet. For example, I use TT-RSS to keep up with Youtube and Reddit (just add .rss to the end of any subreddit URL) as well as the usual blogs and news sites.

About Tiny Tiny RSS

Tiny Tiny RSS is a web based RSS application (think Google Reader replacement). It’s PHP based and supports Postgres or MySQL (like) databases. I’ve been using it for may years and although I’ve tried out other web based RSS readers (such as Miniflux), I’ve never found anything as good as TT-RSS.

tiny tiny rss docker
My Tiny Tiny RSS install

A particular favourite feature of mine is the ability to generate feeds from any internal view, which makes it great for integrating with other systems which may consume RSS/Atom.

There is also an Android app which is available via the Play Store or F-Droid.

Finding a Tiny Tiny RSS Docker Image

I had initially planned to use the LinuxServer.io TT-RSS image, but it seems to have been deprecated. With a bit of searching I found this repo, however it’s pretty out of date and doesn’t build any more. After looking through my GitLab repos, it turned out I’d already tried upgrading that image as part of one of my previous attempts with Docker. I’ve finished off this migration and made the repo public so everyone can benefit from my efforts. Thanks to some CI magic and GitLab’s built in container registry you can pull the latest version like so:

docker pull registry.gitlab.com/robconnolly/docker-ttrss

Feel free to read through the project README to familiarise yourself with the options available in the image. It’s pretty much as it was in the original repository, so I’ll go through my setup below.

Setup with docker-compose

I’m integrating this with my existing Docker setup via docker-compose and Traefik. As such I added the following to my docker-compose.yml file:

ttrss:
    image: registry.gitlab.com/robconnolly/docker-ttrss:latest
    depends_on:
      - mariadb-ttrss
    environment:
      DB_NAME: ${TTRSS_DB_NAME}
      DB_USER: ${TTRSS_DB_USER}
      DB_PASS: ${TTRSS_DB_USER_PASSWD}
      DB_HOST: mariadb-ttrss
      DB_PORT: 3306
      DB_TYPE: mysql
      SELF_URL_PATH: https://ttrss.example.com
    volumes:
      - /mnt/docker-data/ttrss/plugins:/var/www/plugins.local
    labels:
      - 'traefik.enable=true'
      - "traefik.http.middlewares.ttrss_redirect.redirectscheme.scheme=https"
      - "traefik.http.routers.ttrss_insecure.rule=Host(`ttrss.example.com`)"
      - "traefik.http.routers.ttrss_insecure.entrypoints=web"
      - "traefik.http.routers.ttrss_insecure.middlewares=ttrss_redirect@docker"
      - "traefik.http.routers.ttrss.rule=Host(`ttrss.example.com`)"
      - "traefik.http.routers.ttrss.entrypoints=websecure"
      - "traefik.http.routers.ttrss.tls.certresolver=mydnschallenge"
      - "traefik.http.services.ttrss.loadbalancer.server.port=80"
    networks:
      - external
      - internal
    restart: always

Breaking this down, we first create a new service using my TT-RSS image. We then define a dependency on the database container, which we will create later. The environment configuration uses another file env.sh in which we store our secrets. This is of the form:

export TTRSS_DB_ROOT_PASSWD=supersecret
export TTRSS_DB_USER=ttrss
export TTRSS_DB_USER_PASSWD=justalittlebitsecret
export TTRSS_DB_NAME=ttrss

In order to use this the file must be sourced before running docker-compose:

$ source env.sh
$ docker-compose .... # whatever you're doing

We can see that the database is configured entirely via environment variables as shown in the project README. We also set the SELF_URL_PATH variable so that TT-RSS knows where it is located (the URL should be updated for your configuration). I also chose to mount the plugins.local directory on the host machine to allow me to install plugins easily. The remainder of the configuration is for Traefik and is covered in my earlier post (you’ll need to update the hostnames used here too).

Database Setup

As mentioned earlier, we need a database container for TT-RSS to talk to. I’m using MariaDB for this because it’s what I’m familiar with. Also my original TT-RSS installation was in mysql and I wanted to migrate the data. The setup for this is pretty simple using the official MariaDB image:

mariadb-ttrss:
    image: mariadb
    environment:
      MYSQL_ROOT_PASSWORD: ${TTRSS_DB_ROOT_PASSWD}
      MYSQL_USER: ${TTRSS_DB_USER}
      MYSQL_PASSWORD: ${TTRSS_DB_USER_PASSWD}
      MYSQL_DATABASE: ${TTRSS_DB_NAME}
    volumes:
      - /mnt/docker-data/mariadb-ttrss/init/ttrss.sql.gz:/docker-entrypoint-initdb.d/backup.sql.gz
      - /mnt/docker-data/mariadb-ttrss/data:/var/lib/mysql
    networks:
      - internal
    restart: always

As you can see, I re-use the previous environment variables to create the database and user. I also mount the mysql data directory locally and mount a compressed backup of my previous database. This backup will only be loaded the first time the database comes up. You can remove this line if you are doing a clean install.

If you are following my install you’ll also see that I use a couple of Docker networks:

networks:
  external:
  internal:

The external network connects the local service containers to Traefik, whilst the internal network is used between TT-RSS and the database container.

With all this in place you should be able to launch your TT-RSS server with:

docker-compose up -d

At this point, it’s usually a good idea to check the container logs for problems and adjust your configuration accordingly.

Conclusion

Aside from having to update the Docker image for TT-RSS (which took quite a while) this migration was relatively painless. I’m quite happy with my newly Dockerised TT-RSS server. In addition to migrating it into Docker this step has also moved it off my ageing mailserver in preparation for it’s upcoming migration to something newer and moved it from the cloud onto my home server. All positive steps!

Next Steps

I’m pretty keen to keep maintaining my new Docker image for Tiny Tiny RSS, since it seems to be a gap in the community that can be filled. I’m currently building this in CI, but the configuration is pretty basic. In the coming weeks, I’m intending to expand upon this a little and set up a scheduled build which will automatically keep the container up to date. This will hopefully be the topic of a follow up article, so stay tuned!

If you liked this post and want to see more, please consider subscribing to the mailing list (below) or the RSS feed. You can also follow me on Twitter. If you want to show your appreciation, feel free to buy me a coffee.