Tiny MQTT Broker with OpenWRT

So yet again I’ve been really lax at posting, but meh. I’ve still been working on various projects aimed at home automation – this post is a taster of where I’m going…

MQTT (for those that haven’t heard about it) is a real time, lightweight, publish/subscribe protocol for telemetry based applications (i.e. sensors). It’s been described as “RSS for the Internet of Things” (a rather poor description in my opinion).

The central part of MQTT is the broker: clients connect to brokers in order to publish data and receive data in feeds to which they are subscribed. Multiple brokers can be fused together in a heirarchical structure, much like the mounting of filesystems in a unix-like system.

I’ve been considering using MQTT for the communication medium in my planned home automation/sensor network projects. I wanted to set up a heirarchical system with different brokers for different areas of the house, but hadn’t settled on a hardware platform. Until now…

…enter the TP-Link MR3020 ‘travel router’, which is much like the TL-WR703N which I’ve seen used in several hardware hacks recently:

It's a Tiny MQTT Broker!

It’s a Tiny MQTT Broker!

I had to ask a friend in Hong Kong to send me a couple of these (they aren’t available in NZ) – thanks Tony! Once I received them installing OpenWRT was easy (basically just upload through the exisiting web UI, follow the instructions on the wiki page I linked to above). I then configured the wireless adapter in station mode so that it would connect to my existing wireless network and added a cheap 8GB flash drive to expand the available storage (the device only has 4MB of on-board flash, of which ~900KB is available after installing OpenWRT). I followed the OpenWRT USB storage howto for this and to my relief found that the on-board flash had enough space for the required drivers (phew!).

Once the hardware type stuff was sorted with the USB partitioned (1GB swap, 7GB /opt) and mounting on boot, I was able to install Mosquitto, the Open Source MQTT broker with the following command:

$ opkg install mosquitto -d opt

The -d option allows the package manager to install to a different destination, in this case /opt. Destinations are configured in /etc/opkg.conf.

It took a little bit of fiddling to get mosquitto to start at boot, mainly because of the custom install location. In the end I just edited the paths in /opt/etc/init.d/mosquitto to point to the correct locations (I changed the APP and CONF settings). I then symlinked the script to /etc/rc.d/S50mosquitto to start it at boot.

That’s about as far as I’ve got, apart from doing a quick test with mosquitto_pub/mosquitto_sub to test everything works. I haven’t tried mounting the broker under the master broker running on my home server yet.

The next job is to investigate the serial port on the device in order to attach an Arduino clone which I soldered up a while ago. That will be a story for another day, hopefully in the not-to-distant future!

Smartclock Prototype

So as promised here are the details and photos of the Arduino project I’ve been working on – a little late I know, but I’ve actually been concentrating on the project.

The project I’m working on is a clock, but as I mentioned before it’s not just any old clock. The clock is equipped with sensors for temperature, light level and battery level. It also has a bluetooth module for relaying this data back to my home server. This is the first part of a larger plan to build a home automation and sensor network around the house (and garden). It’s serving as kind of a test bed for some of the components I want to use as well as getting me started with the software.

Prototype breadboard

The prototype breadboard showing the Roving Networks RN-41 bluetooth module on the left and the sensors on the right. The temperature sensor (bottom middle) is a TMP36 and the light sensor is a simple voltage divider using a photocell.

As you can see from the photos this is a very basic prototype at the moment – although as of this weekend all the hardware is working as well as the software drivers. I just have the firmware to finalise before building the final unit.

Time display

The (very bright!) display showing the time. I’m using the Sparkfun 7-segment serial display, which I acquired via Nicegear. It’s a lovely display to work with!

The display is controlled via SPI and the input from the light sensor is used to turn off the display when it is dark in order to save power when there is no-one in the room. The display will also be able to be controlled from the server via a web interface.

Temperature display

The display showing the current temperature. The display switches between modes every 20 seconds with it’s default settings.

Careful readers will note the absence of a real time clock chip to keep accurate time. The time is kept using one of the timers on the ATmega328p. Yes, before you ask this isn’t brilliantly accurate (it loses about 30 seconds every hour!), but I am planning to sync the time from the server via the bluetooth connection, so I’m not concerned.

The final version of will use an Arduino Pro Mini 3.3v (which I also got from Nicegear) for the brains, along with the peripherals shown. The Duemilanove shown is just easier for prototyping (although it makes interfacing with the RN-41 a little more difficult).

I intend to publish all the code (both for the firmware and the server) and schematics under Open Source licences as well as another couple of blog posts on the subject (probably one on the final build with photos and one on the server). However, that’s it for now.

Quick Update…

Well, I’ve not been doing great with posting more, especially on the quick short posts front. I guess it’s because I’m either too busy all the time or because I just don’t think anyone wants to read every last thought which pops into my head. Probably a bit of both!

Anyway, here’s a quick run-down of what I’ve been up to over the past couple of weeks:

  • I’ve been working on an Arduino project at home. I’ll post more on this over the weekend (with photos). For now I’ll just say that it’s a clock with some sensors on it – but it does a little more than your average clock. Although I’ve had my Arduino for a couple of years I’ve never really used it in earnest and I’m finding it refreshing to work with. Since I use PICs at work the simpler architecture is nice. Of course I program it in C so I can’t comment on the IDE/language provided by the Arduino tools.
  • The beer which I made recently is now bottled and maturing. It’ll need a couple more weeks to be ready to drink though (actually the longer the better really, but I can never wait!). I’ll report back on what it’s like when I try it.
  • I’ve been thinking about ways to get the ton of data I have spread across my machines in order. Basically I want to get it all onto my mythbox/home server/personal cloud and acessible via ownCloud and NFS. I also have a ton of dead tree (read ‘important’ documents) which need scanning and a ton of CDRs that need backing up. After that I have to overhaul my backup scheme. It’s a big job – hence why I’ve only been thinking about doing it so far.
  • I’ve also been thinking about upgrading my security with the recent hacks which have occured. Since I’m not hugely reliant on external services (i.e. Google, Facebook, Apple and Amazon) I’m doing pretty well already. Also, I already encrypt all my computers anyway (which is way more effective than that stupid ‘remote wipe’ misfeature Apple have). I am considering upgrading to two factor authentication using Google Authenticator anywhere I can and I want to switch to using GPG subkeys and storing my master private key somewhere REALLY secure. I’ll be writing about these as I do them so stay tuned.

Well, hopefully that’s a quick summary of what I’ve been up to (tech-wise) lately as well as what might be to come in these pages. For now, that’s all folks.

UoA ECE Department ‘DonKey’ on Linux

Introduction: Since finishing my Part IV Project, I’ve been threatening to do some embedded/microcontroller stuff in my spare time at home. I’ve now finally go around to it and I thought I’d start by playing with a few components I had left over from a Uni project a while back. I’ve also ordered an Arduino board (see below), but it hasn’t arrived yet. When it does, I think I’m going to have a go programming it in C rather than the random Arduino language, as I have much more experience of programming embedded systems than your average Arduino user. I’ll report on my progress when I have some!

In the Electrical and Computer Engineering Deaprtment of the University of Auckland, where I work we have a little device, internally known as the ‘DonKey’. The purpose of this is to allow easy programming of Atmel AVR based microcontrollers via USB, rather than the simpler serial interface. We also have some internally developed software to program microcontrollers via the device, unfortunately this software is pretty much windows only (we did have a successful attempt to compile it for Linux, but this was quite a while ago, a better solution would be to use a native Linux application).

Internally the DonKey uses an FTDI based USB to UART chip (specifically the FT232R) to communicate with the microcontroller. This presents some problems as, despite being the basis of the programmer on incredibly popular Arduino boards, the main Linux programming tool (AVRdude) has no official FTDI support. I think this is largely due to the use of a bootloader on the Arduino boards, which negates the need of the programming tool to directly flash the board. If however you brick the AVR on the Arduino, you would be out of luck and would need a physical programmer (more on this below).

The DonKey

The DonKey in all it's glory.

In this howto I’ll cover how to get the DonKey working on Linux with AVRdude. Luckily, while researching how I might go about this I found that a large part of the work had been done for me, due to the fact that the Arduino also uses these chips. I found instructions on doswa.com on how to patch and compile AVRdude for just this purpose (so you could flash a bootloader to a new AVR).

These instructions work quite well for the DonKey, up until you get to running the ‘./configure’ command, I replaced this with:

$ ./configure --prefix=$HOME/.local

to setup the code to do a local install in my home directory (as I want this to be my primary version of AVRdude, but not to screw with things on the root filesystem).

Next I followed the instructions on modifiying the makefile and compiling AVRdude via the ‘make’ command. After ‘make’ I also typed:

$ make install

to install into the directory setup earlier. Now AVRdude is installed, the next thing to do is a bit of configuration, firstly you’ll want to make sure it’s on your $PATH so add the following to your ~/.bashrc file:

export PATH=$HOME/.local/bin:$PATH

and run the command:

$ source ~/.bashrc

to re-read the file.

The next issue is that you may wish to remove any copy of AVRdude that is otherwise installed (I found that sometimes my shell would run the wrong one – especially if you use ‘sudo’ to run it):

sudo apt-get remove --purge avrdude

Now, I just mentioned above that you might use ‘sudo’ to run AVRdude, well according to the doswa article you do need to use sudo when using the FTDI based programmers. I’m not sure why this is, but it’s not very useful if you want to be able to call AVRdude from a Makefile or the like.

I solved this by setting a ‘suid root’ on my AVRdude binary. For those that don’t know what this is, the suid bit is a Unix permission setting that makes any program with it run under it’s owning user rather than the user who called it. If the owner happens to be root, the program runs as root even if the user who calls it isn’t. This is probably really insecure if you do it a lot, but you should be OK in this case.

WARNING: Despite what I say, it might not be OK. Allowing any program unrestricted root access has the potential to hose your system and scatter all your data to the winds. FOLLOW THESE INSTRUCTIONS AT YOUR OWN RISK!!

So here we go:

$ sudo chown root:root ~/.local/bin/avrdude
$ sudo chmod u+s ~/.local/bin/avrdude

Now you should be able to successfully run AVRdude on FTDI based devices without resorting to using sudo every time.

But, what of the DonKey I hear you cry! Well all we have to do to support the DonKey is give AVRdude a little bit of configuration which tells it what the DonKey actually is. This can go in ~/.avrduderc, and looks a bit (well exactly) like this:

id = "donkey";
desc = "University of Auckland ECE DonKey";
type = ft245r;
miso = 1; # D1
sck = 2; # D2
mosi = 3; # D3
reset = 4; # D4

OK, now you should be able to successfully use the DonKey with AVRdude, using a command similar to this:

avrdude -c donkey -p m8 -P ft0 -U myawesomeavrproject.hex

Note: this command is for the ATMega8 as denoted by the ‘-p m8’, check the AVRdude manual page for the correct -p option if you are using a different type of AVR.

OK, well that’s pretty much it, I’ll post back soon regarding my other progress with some microcontroller stuff. Bye for now!