mysensors greenhouse sensor

Prototyping Battery Powered Sensors with MySensors and Home Assistant

This post may contain affiliate links. Please see the disclaimer for more information.

Sometimes it’s nice to be able to undertake a spur of the moment project that can be completed in a couple of hours. Of course, this rarely happens if the project is hardware related due to the need to wait for any parts you may not have to hand. This article will relate one such rare occurrence in my decision to try out the MySensors project for an outdoor, battery powered sensor prototype which would be integrated with Home Assistant.

The motivation for this project was the installation of a small greenhouse kit from the local hardware store, so that we had somewhere sheltered to raise seeds for the garden. The positioning of the greenhouse is along the back wall of our house, which gets quite a bit of sun. I realised it would be nice to log the temperature inside the greenhouse and send notifications when it was outside certain thresholds. I could then go out to open the door and otherwise check on the seedlings. As you will see below, I was able to build such a sensor very quickly and get it running using pretty much unmodified example code.

About MySensors

MySensors is an Arduino library for building low power mesh networked sensors based NRF24L01+ and RFM69 radios. It allows you to easily get the networking up and running so you can focus on building and deploying your sensors.

The sensors connect back to a gateway device which forwards traffic on to some controller software. There are several different types of gateway available, ranging from a simple serial gateway to networked options using MQTT and the ESP8266. There are also several different controllers available. However, I’ll be focussing on using Home Assistant.

Since these sensors are based on relatively low powered components, such as low bit-rate radios and 8-bit Arduinos they are much more suited to battery operation than the equivalent ESP8266 or ESP32 based sensor. The trade off for this is that the performance and capabilities are lower. For that reason I’d advise you to use an ESP based system with ESPHome if you have a decent power supply available or where you are actuating devices that already have higher power requirements. This is because MySensors actuation devices are not allowed to sleep in order to receive commands, which negates many of the power savings.

Parts and Assembly

For my initial deployment I wanted to test the power usage of a sensor in the field so as to see if this approach is viable. This sensor would use a DHT11 temperature and humidity sensor along with the radio and would be powered from 2xAA batteries. I also built a simple serial gateway which I connected directly to my main server via USB.

mysensors home assistant
The assembled gateway
mysensors home assistant
The assembled electronics for the sensor

The parts used in my build were as follows:

mysensors case
The sensor case, fresh off the printer
mysensors case
Packing the electronics in the case resulted in quite the birds nest
mysensors case
However, everything looked neat with the lid on!

These were assembled pretty much as per the instructions. First I connected up the radios to both gateway and sensor. I then connected up the DHT sensor and the battery holder on the sensor side. It’s worth noting that I used digital pin 4 rather than pin 3 for the DHT because I connected it without looking at the instructions. However, it makes little difference, just a minor change to the example code.

I didn’t make any hardware optimisations for low power usage at this stage, except for the obvious one of removing the power LED on the pro mini board (LED1 on the schematic). This will save a significant amount of power, since otherwise it would be on all the time.

Software

The software side of things was even more trivial than the hardware. For the gateway I just flashed the provided serial gateway code. For the sensor I just flashed the DHT example code with the small modification to the pin noted above. Both of these just worked without any further tweaking. The DHT example did require me to include a custom DHT sensor library which I copied from the git repo in question.

I used my preferred tool of Platformio for building and flashing. The only things to consider here are to initialise the projects with the correct board names (diecimilaatmega328 and pro8MHzatmega328 in my case) and to install the MySensors library in each project. This can be accomplished with the command:

platformio lib install --save MySensors

If you want to reproduce my exact setup or use it as a basis for your own, you can grab my code from GitLab.

Connecting MySensors to Home Assistant

With the firmware flashed to the devices, the next step is setting up the MySensors controller in Home Assistant. The first step in this process was making the serial adapter of the Arduino available through the layers of virtualisation in my setup. In order to do this I first added the serial adapter as a USB device to the KVM virtual machine via virt-manager. Secondly, I added the device to the Home Assistant container in my docker-compose.yml file with the entry:

devices:
  - /dev/ttyUSB0:/dev/ttyUSB0

Once the deployment had finished, I was able to add the MySensors integration to my Home Assistant configuration, which I did in a new package file:

mysensors:
   gateways:
     - device: /dev/ttyUSB0
       persistence_file: '/config/mysensors.json'
   debug: true
   persistence: true
   version: '2.3'
   optimistic: false

This sets up the integration with a single serial gateway on the port in question. With HASS restarted after this change, I was able to see my sensors. However, the entity IDs and resulting friendly names are based on the MySensors internal identifiers. I don’t really mind the cryptic entity IDs, but I updated the friendly names by adding the following to my customize section:

sensor.temperatureandhumidity_3_0:
friendly_name: "Greenhouse Humidity"

sensor.temperatureandhumidity_3_1:
friendly_name: "Greenhouse Temperature"

Notifications

The final step in the HASS setup for this achieves the goal of being notified when I need to check on the seedlings. This was accomplished via a couple of simple automations:

automation:
  - alias: "Greenhouse High Temperature Notification"
    trigger:
      platform: numeric_state
      entity_id: sensor.temperatureandhumidity_3_1
      above: 27
      for:
        minutes: 30
    condition:
      condition: state
      entity_id: input_boolean.greenhouse_active
      state: 'on'
    action:
      service: notify.family
      data_template:
        title: "Greenhouse Temperature High!"
        message: "The greenhouse temperature is {{ states.sensor.temperatureandhumidity_3_1.state }}{{ states.sensor.temperatureandhumidity_3_1.attributes.unit_of_measurement }}. You may wish to open the door and check if the plants need watering."
  - alias: "Greenhouse Low Temperature Notification"
    trigger:
      platform: numeric_state
      entity_id: sensor.temperatureandhumidity_3_1
      below: 11
      for:
        minutes: 30
    condition:
      condition: state
      entity_id: input_boolean.greenhouse_active
      state: 'on'
    action:
      service: notify.family
      data_template:
        title: "Greenhouse Temperature Low!"
        message: "The greenhouse temperature is {{ states.sensor.temperatureandhumidity_3_1.state }}{{ states.sensor.temperatureandhumidity_3_1.attributes.unit_of_measurement }}. You may wish to close the door."

These are pretty simple and send the notifications via my Gotify server. The only minor bit of complexity was that I added an input boolean to allow the notifications to be controlled together. I could just as easily turn off both automation rules when the greenhouse is not in use, but it’s nice to have a single switch for that purpose:

input_boolean:
  greenhouse_active:
    name: "Greenhouse In Use"
    icon: mdi:sprout

Range Issues

With all that in place, I tested the sensor overnight inside the house and then deployed it the following morning. I had placed it inside as close to the back wall where the greenhouse is located as possible and it worked fine. It even worked fine after deploying it in the greenhouse for around 12 hours.

mysensors greenhouse sensor
The sensor deployed inside the greenhouse (from inside)
mysensors home assistant
From outside the plastic of the greenhouse protects the electronics from the elements

Then it stopped sending data.

I brought the sensor in and examined it. The battery voltage was fine and after a reset it came back up, so I deployed it again. The same thing happened within a few minutes. I then brought the sensor back inside and put it in the garage, just a few meters from the gateway. It’s been operating there successfully for several days.

I can only conclude from this that the radio range is pretty poor. I have a couple of mitigations in mind for this, the first of these will be to move the gateway closer to the garden. For this I may need a networked gateway, or I may plug it into a nearby Raspberry Pi. I’ll try the Raspberry Pi idea first and see if I can get it working with the Node-RED MySensors nodes. Ideally, I’d like these just to operate as a proxy so that the MySensors controller will still be Home Assistant.

The other potential mitigation is to use a radio with a higher gain antenna and amplifier on the gateway. I may do this anyway in time in order to get better range into the garden for other sensors.

Conclusion

Overall I’m pretty pleased with the MySensors setup I’ve been able to create here. It’s very much a prototype and there are still a few issues to work out, but it’s promising. The best thing is that it only took me a couple of hours, though this was aided massively by having the parts already to hand.

MySensors itself seems quite nice and there is certainly a large community around it. As compared to other offerings such as ESPHome, it feels a bit basic to actually have to write (or copy) Arduino code and upload it to the device. I much prefer the YAML approach for describing simple sensors. That said, I still had to write no code in order to get this working! I’d love to see a similar YAML based code generator for this platform though.

I’ll definitely be building and deploying a few more MySensors nodes, primarily in the garden where ability to run off batteries for a decent length of time is a great advantage. However, I need to fix the range issue first, which I’ll be sure to post an update on.

As always please get in contact to share your own experiences!

If you liked this post and want to see more, please consider subscribing to the mailing list (below) or the RSS feed. You can also follow me on Twitter. If you want to show your appreciation, feel free to buy me a coffee.

Loading

My Home Automation Setup

Welcome to this long awaited (by me at least) post about my Home Automation setup, specifically my Home Assistant configuration. It feels like I’ve been promising this post forever, so it’s great to finally deliver. Let’s get straight into it.

I’ve pushed my full Home Assistant configuration to GitLab and will be linking to various parts throughout this post. The config makes heavy use of secrets, to obscure sensitive details about my infrastructure (the sensitive files themselves are encrypted with git-crypt).

What is Home Assistant?

Home Assistant Logo

 

Unless you’ve had your head in the sand or just have no interest in home automation (why are you reading this?) you should know that Home Assistant (HASS) is an Open Source home automation server written in Python (3). What you might not know is that it has an awesome and very active set of developers, two weekly release cycle and a friendly and helpful community. At the time of writing it supports 668 different components. A component is the HASS term for an integration with an external device, tool or service. These components form the base for a device and manufactuer agnostic home automation system. Due to the sheer number of them, chances are you have several HASS compatible devices or service subscriptions, even if you are not into home automation.

I ended up trying out Home Assistant whilst looking for an alternative to OpenHAB. My reasons for switching away from OpenHAB were that I found the profusion of DSLs unweildy and the automation rules difficult to debug, so I never managed to get a satisfactorily working system. I haven’t followed the project since moving to HASS, so these issues might have improved. I would encourage anyone for whom HASS doesn’t appeal to head over and take a look.

After being initally sceptical that the YAML based configuration and automations in HASS would be flexible enough to do what I wanted, I was pleasently surprised by the level of functionality available at little effort. Things like Scenes, which had been tricky in OpenHAB, were easy in HASS! Overall Home Assistant just operates at a higher level of abstraction, which makes things simpler. Since I switched the project has gone from strength to strength and gets better with every release!

My Setup and Hardware

My Home Assistant Frontend

My Home Assistant Frontend

As previously documented Home Assistant runs in a VM on my main home server. It talks mostly to devices in an isolated subnet – so that my IoT devices are firewalled from both the main LAN and the internet at large. The firewall on this subnet blocks both incoming and outgoing traffic, preventing any of these devices from calling home without my say so. The only ports open are those required to allow the devices to speak to HASS or the MQTT server (plus DNS and DHCP to the firewall).

In terms of hardware, I currently have:

  • 3x Milight GU10 RGBW bulbs and the associated (crappy) wifi bridge (I actually have the older wifi bridge). These are located in the living room and provide the main overhead lighting as well as evening mood lighting when watching TV.
  • 2x Kankun SP3 wifi switches, for which I wrote the HASS platform integration. These are connected to some low power (500W) heaters which provide heating in the bedrooms.
  • 2x Raspberry Pis with various sensors connected (temperature/humidity and motion). These also have attached Webcams which stream to ZoneMinder via MJPG-Streamer. One of these also forms the RC-Switch gateway for the RF switches below.
  • 3x 433Mhz dumb RF switches connected to a couple of lamps and the kettle, these are actuated via an MQTT to RC-Switch bridge implemented in Node-RED running on the aforementioned RPi.
  • 2x home built sensors which publish sensor data (temperature, light and motion) from the bedrooms via MQTT.
  • 1x Cheap (and insecure) nondescript Chinese IP camera – yes, I did change the default password and this thing is well firewalled (incoming and outgoing). Again, this streams to ZoneMinder.

In addition, I have Home Assistant setup with my Kodi instance as a media player and use the DarkSky support for providing weather information. Presence detection is provided via OwnTracks and SNMP from my pfsense firewall.

For notifications, I’m using both SMTP and the native HTML5 notifications. I’m particularly pleased with the HTML5 notifications, which are a recent addition to my config. They are a bit of a pain to setup, but they finally spurred me on to getting a proper nginx reverse proxy set up for HASS and getting a proper TLS certificate with Let’s Encrypt. Once set up they a reliable, fast and look great on all our devices. I haven’t yet tried out the actions support, but it does open up some interesting possibilities for new automations.

One component which is conspiciously absent from my setup is any kind of physical control of the smart devices. Some devices (like the Kankun switches) will sync their state back to HASS when manually actuated, but most need to be directly addressed by HASS in order to prevent getting into an inconsistent state. Aside from implementing automations (detailed below), so that manual intervention isn’t required for common actions, we also use the HTTP Shortcuts app on our phones for reasonably quick switching of devices. The user experience is still not brilliant, so it would be good to have some physical switches for some things in the future.

Scripts and Automations

The ‘A’ in Home Automation is for automation. Up until now, I’ve described the infrastructure that provides me with (remote) Home Control. The real power of HASS is in its capability to act without direct user input, or to react to your behaviour (somewhat) intelligently. To this end I have several automations and scripts set up to perform various actions, based on what’s going on. In no particular order these are:

  • Execute an ‘Away Mode’ script when presence detection reports everyone has left. There is a corresponding ‘Deactivate Away Mode’ script for when someone returns home. The away mode script switches my cameras into motion detect mode (indoor cameras are also powered up, since they are powered down for privacy when we are home). It also sends a notification to tell us this has happened.
  • On weekday mornings turn on the lights if it is still dark outside and turn on the kettle (this will cause it to boil automatically if I remembered to set it up the night before). The lights get turned off a little while later. The kettle gets turned off at night, ready for the next day. This automation only executes if someone is home.
  • Switch the living room lights into Movie mode in the evening (after a certain time and if the sun has set), when pressing play in Kodi. If playback is paused or stopped the main lights come back up. I recently added transitions to these changes, which don’t work perfectly with the Milight bulbs but are better than the harsh change I had before.
  • Turn on the lights just before sunset, with an adjustment for cloud cover. Through a bit of experimentation I found that 20 mins before sunset was good for fine days and 40 mins was good for cloudy days. This automation only executes if someone is home.
  • Turn on the lights if someone comes home after sunset. This works really well with the combination presence detection via OwnTracks and SNMP – the lights are always on by the time we get in the house.
  • Send various notifications, mostly for motion detection if we are out or at certain times. Also sends me an email when HASS has an upgrade pending.
  • This isn’t really an automation, but I use the Generic Thermostat component to control the heaters in the bedrooms. This is filtered to switch off based on presence by a couple of template sensors which wrap the actual temperature sensors when someone is home and return 100°C when everyone is out, thus making sure the heaters stay off. I’m sure there is a better way to do this, but it works.
  • I also run AppDaemon, but currently only have a single app – OccuSim, which mimics our lighting patterns when we are away.

That’s pretty much it. Those automations cover most of our current use cases, but there are definately more to come and there will be much more as I add more devices to the setup.

The Future

There’s lots to add in future. In addition to solving the manual input problems, I’m very interested in replacing the unreliable RC-Switches with some Sonoff modules (running custom firmware) and adding some more for other appliances. I’m also interested in upgrading my Milight gateway to a homebuilt one based on the ESP8266/NRF24L01 modules. I also like the look off the Yeelight bulbs as a better alternative to Milight. In addition I’ll soon be moving my Kodi install to a Raspberry Pi which will allow me to play with some HDMI-CEC stuff. I also need to improve the design of my frontend pages in HASS.

I’ve looked into many of the available components within HASS for devices which may be applicable to my setup, including Z-Wave and the Zigbee based platforms (Hue, IKEA Tradfri, etc.). Many of these platforms either don’t meet my requirements in terms of security (most Z-Wave devices don’t appear to use encryption [not that Milight or RC-Switch are secure either, but at least they are only switching low power LEDs]), are too expensive (looking at you Hue) or have limited availability here in NZ (Z-Wave devices using the AU/NZ spectrum are limited in number and I can’t find anywhere shipping the Tradfri devices to NZ [for the record Tradfri fits my security requirements]). For these reasons, I’ve been working on more custom devices based on Arduino/MySensors (for battery powered devices) and ESP8266/Homie for mains powered devices. The key here will be building something reliable and getting the physical design right so it is robust and not unsightly. Some of the new Z-Wave Plus devices also look promising due to their encryption support (particularly the Aeotec devices e.g. this).

For all of the above, I’ll try to do write ups here when I can. It’s a pretty long list and should keep me going for quite some time!

I hope you’ve enjoyed reading this high level view of my home automation setup. I’ll happily receive suggestions for improvements in the comments, or field questions. You can follow me on Twitter if you are that way inclined or subscribe to the RSS or email updates (see sidebar). Thanks again for reading.

Tiny MQTT Broker with OpenWRT

This post may contain affiliate links. Please see the disclaimer for more information.

So yet again I’ve been really lax at posting, but meh. I’ve still been working on various projects aimed at home automation – this post is a taster of where I’m going…

MQTT (for those that haven’t heard about it) is a real time, lightweight, publish/subscribe protocol for telemetry based applications (i.e. sensors). It’s been described as “RSS for the Internet of Things” (a rather poor description in my opinion).

The central part of MQTT is the broker: clients connect to brokers in order to publish data and receive data in feeds to which they are subscribed. Multiple brokers can be fused together in a heirarchical structure, much like the mounting of filesystems in a unix-like system.

I’ve been considering using MQTT for the communication medium in my planned home automation/sensor network projects. I wanted to set up a heirarchical system with different brokers for different areas of the house, but hadn’t settled on a hardware platform. Until now…

…enter the TP-Link MR3020 ‘travel router’, which is much like the TL-WR703N which I’ve seen used in several hardware hacks recently:

It's a Tiny MQTT Broker!

It’s a Tiny MQTT Broker!

I had to ask a friend in Hong Kong to send me a couple of these (they aren’t available in NZ) – thanks Tony! (UPDATE 2019: Of course now you can get these shipped direct, something I didn’t know about in 2012). Once I received them installing OpenWRT was easy (basically just upload through the exisiting web UI, follow the instructions on the wiki page I linked to above). I then configured the wireless adapter in station mode so that it would connect to my existing wireless network and added a cheap 8GB flash drive to expand the available storage (the device only has 4MB of on-board flash, of which ~900KB is available after installing OpenWRT). I followed the OpenWRT USB storage howto for this and to my relief found that the on-board flash had enough space for the required drivers (phew!).

Once the hardware type stuff was sorted with the USB partitioned (1GB swap, 7GB /opt) and mounting on boot, I was able to install Mosquitto, the Open Source MQTT broker with the following command:

$ opkg install mosquitto -d opt

The -d option allows the package manager to install to a different destination, in this case /opt. Destinations are configured in /etc/opkg.conf.

It took a little bit of fiddling to get mosquitto to start at boot, mainly because of the custom install location. In the end I just edited the paths in /opt/etc/init.d/mosquitto to point to the correct locations (I changed the APP and CONF settings). I then symlinked the script to /etc/rc.d/S50mosquitto to start it at boot.

That’s about as far as I’ve got, apart from doing a quick test with mosquitto_pub/mosquitto_sub to test everything works. I haven’t tried mounting the broker under the master broker running on my home server yet.

The next job is to investigate the serial port on the device in order to attach an Arduino clone which I soldered up a while ago. That will be a story for another day, hopefully in the not-to-distant future!

If you liked this post and want to see more, please consider subscribing to the mailing list (below) or the RSS feed. You can also follow me on Twitter. If you want to show your appreciation, feel free to buy me a coffee.

Loading

Smartclock Prototype

So as promised here are the details and photos of the Arduino project I’ve been working on – a little late I know, but I’ve actually been concentrating on the project.

The project I’m working on is a clock, but as I mentioned before it’s not just any old clock. The clock is equipped with sensors for temperature, light level and battery level. It also has a bluetooth module for relaying this data back to my home server. This is the first part of a larger plan to build a home automation and sensor network around the house (and garden). It’s serving as kind of a test bed for some of the components I want to use as well as getting me started with the software.

Prototype breadboard

The prototype breadboard showing the Roving Networks RN-41 bluetooth module on the left and the sensors on the right. The temperature sensor (bottom middle) is a TMP36 and the light sensor is a simple voltage divider using a photocell.

As you can see from the photos this is a very basic prototype at the moment – although as of this weekend all the hardware is working as well as the software drivers. I just have the firmware to finalise before building the final unit.

Time display

The (very bright!) display showing the time. I’m using the Sparkfun 7-segment serial display, which I acquired via Nicegear. It’s a lovely display to work with!

The display is controlled via SPI and the input from the light sensor is used to turn off the display when it is dark in order to save power when there is no-one in the room. The display will also be able to be controlled from the server via a web interface.

Temperature display

The display showing the current temperature. The display switches between modes every 20 seconds with it’s default settings.

Careful readers will note the absence of a real time clock chip to keep accurate time. The time is kept using one of the timers on the ATmega328p. Yes, before you ask this isn’t brilliantly accurate (it loses about 30 seconds every hour!), but I am planning to sync the time from the server via the bluetooth connection, so I’m not concerned.

The final version of will use an Arduino Pro Mini 3.3v (which I also got from Nicegear) for the brains, along with the peripherals shown. The Duemilanove shown is just easier for prototyping (although it makes interfacing with the RN-41 a little more difficult).

I intend to publish all the code (both for the firmware and the server) and schematics under Open Source licences as well as another couple of blog posts on the subject (probably one on the final build with photos and one on the server). However, that’s it for now.

Quick Update…

Well, I’ve not been doing great with posting more, especially on the quick short posts front. I guess it’s because I’m either too busy all the time or because I just don’t think anyone wants to read every last thought which pops into my head. Probably a bit of both!

Anyway, here’s a quick run-down of what I’ve been up to over the past couple of weeks:

  • I’ve been working on an Arduino project at home. I’ll post more on this over the weekend (with photos). For now I’ll just say that it’s a clock with some sensors on it – but it does a little more than your average clock. Although I’ve had my Arduino for a couple of years I’ve never really used it in earnest and I’m finding it refreshing to work with. Since I use PICs at work the simpler architecture is nice. Of course I program it in C so I can’t comment on the IDE/language provided by the Arduino tools.
  • The beer which I made recently is now bottled and maturing. It’ll need a couple more weeks to be ready to drink though (actually the longer the better really, but I can never wait!). I’ll report back on what it’s like when I try it.
  • I’ve been thinking about ways to get the ton of data I have spread across my machines in order. Basically I want to get it all onto my mythbox/home server/personal cloud and acessible via ownCloud and NFS. I also have a ton of dead tree (read ‘important’ documents) which need scanning and a ton of CDRs that need backing up. After that I have to overhaul my backup scheme. It’s a big job – hence why I’ve only been thinking about doing it so far.
  • I’ve also been thinking about upgrading my security with the recent hacks which have occured. Since I’m not hugely reliant on external services (i.e. Google, Facebook, Apple and Amazon) I’m doing pretty well already. Also, I already encrypt all my computers anyway (which is way more effective than that stupid ‘remote wipe’ misfeature Apple have). I am considering upgrading to two factor authentication using Google Authenticator anywhere I can and I want to switch to using GPG subkeys and storing my master private key somewhere REALLY secure. I’ll be writing about these as I do them so stay tuned.

Well, hopefully that’s a quick summary of what I’ve been up to (tech-wise) lately as well as what might be to come in these pages. For now, that’s all folks.