rss-bridge

Reconnecting the Web with RSS-Bridge

This post may contain affiliate links. Please see the disclaimer for more information.

I’ve mentioned before that I’m a big fan of RSS as a medium for consuming my daily news and for following the blogs of others. However, there are an increasing number of websites that don’t provide an RSS feed (or at least don’t advertise a feed if one exists). Luckily for us there is an awesome piece of self-hosted software which aims to fill in the gaps left by these missing feeds – RSS-bridge.

My use case for this was twofold. First I wanted to follow some sites for which I couldn’t find RSS feeds, specifically The Guardian. Second, I wanted to get updates from some local groups, who only have a Facebook page. Obviously, I don’t actually want to actually check in to Facebook to do this, that would be intolerable. RSS-Bridge fills both these needs.

Installation

There are several public instances of RSS-bridge available, but of course I wanted to host my own. Doing so is extremely easy with Docker. I added the following to my docker-compose.yml file on the server in question:

services:
  rss-bridge:
    image: rssbridge/rss-bridge:latest
    volumes:
      - /mnt/docker-data/rss-bridge/whitelist.txt:/app/whitelist.txt
    labels:
      - 'traefik.enable=true'
      - "traefik.http.middlewares.rssbridge_redirect.redirectscheme.scheme=https"
      - "traefik.http.routers.rssbridge_insecure.rule=Host(`rssbridge.example.com`)"
      - "traefik.http.routers.rssbridge_insecure.entrypoints=web"
      - "traefik.http.routers.rssbridge_insecure.middlewares=rssbridge_redirect@docker"
      - "traefik.http.routers.rssbridge.rule=Host(`rssbridge.example.com`)"
      - "traefik.http.routers.rssbridge.entrypoints=websecure"
      - "traefik.http.routers.rssbridge.tls.certresolver=mydnschallenge"
      - "traefik.http.services.rssbridge.loadbalancer.server.port=80"
    networks:
      - external
    restart: always

This uses Traefik, with my internal HTTPS setup to serve the bridge over HTTPS. You can also set up authentication for the bridge if you like. This isn’t really required unless you are hosting the bridge on a publicly available URL and would rather keep it private. I elected not to bother with authentication, since mine is on my internal network. It should also be noted that the bridge is totally stateless. All the parameters are sent in the URL, so there is no data to protect.

Grabbing Feeds

You’ll see above that we mounted a text file called whitelist.txt inside the container. This contains a list of all the bridges you want to use, from the full list of bridges. Here’s mine:

FacebookBridge
TheGuardianBridge
TwitterBridge
YoutubeBridge

I’ll demonstrate the use of a couple of these below, but it’s pretty simple. First up TheGuardianBridge, just select the section of the site you are interested in and click a button – couldn’t be easier!

rss-bridge
Super simple!

I like to use the HTML button so that I can see that the bridge is working right there in the browser. You can then grab the (M)RSS or Atom links directly from the resulting page:

rss-bridge
The resulting feed page

I’m also going to grab a feed of my local council news from their Facebook page, using the FacebookBridge:

rss-bridge
The Facebook Bridge

Here we just enter the name of the page or user we are interested in. There is another dialogue below this for groups, but I haven’t tried that yet. I assume this only works for public pages, since it doesn’t ask for any login credentials. Of course, when we click through we are greeted by our feed:

rss-bridge
The resultant Facebook feed

The Twitter bridge works similarly. I haven’t had much luck with the Youtube bridge, but I’m already using a well known trick to get RSS feeds of my favourite Youtube channels.

Setting Up Email Notifications

So far, this has all been very easy. Let’s step it up (just a little) and get notified when one of our feeds gets updated. I’m using this to be notified of events and goings on in my local area via some of the Facebook feeds. This closes the loop quite nicely and takes “social media” back to the promise it had in the early days.

To do this I’m using a tool called rss2email. This is a brilliant little tool, which I actually used as my primary RSS reader for some years, until I got too many feeds to get through all the emails! I’m glad to press it back into service for this.

I elected not to install rss2email in Docker, since I couldn’t find a nicely updated image and didn’t fancy building my own. It’s also kind of a personal tool, so fits nicely in a Unix user account as a cron job. On Ubuntu rss2email can be installed via APT:

$ sudo apt install rss2email

Next it’s best to follow the official documentation to get it up and running. You’ll need some access to an SMTP server to be able to send mail. One place where the documentation seems to differ is in enabling SMTP, where I had to use the line email-protocol = smtp rather than the use-smtp specified in the docs.

Once this is all set up you can add your feeds like so:

$ r2e add FeedName https://rss-bridge.example.com/.....

Of course you can add non-RSS-bridge feeds too. Just add whatever feeds you’d like to receive notifications on!

The last thing is to schedule this as a cron job:

14  *  *   *   *     /usr/local/bin/log-output "/usr/bin/r2e run"

I’m using the wrapper script I’ve mentioned previously. Done!

Conclusion

This has been a really simple project (by my standards). Everything went according to plan, which almost never happens! Regardless, I’m very happy with the result and it’s something I’ll continue to make use of every day.

RSS-Bridge fills a much needed hole in the modern web. With the dominance of the big social media platforms and increasing “appification”, we’ve lost the real promise of the web to be an open and connected platform. RSS-bridge brings back at least some of this.

The addition of rss2email fulfils the hopes I had for social networks in the early days – that they would become notification platforms for events/people/things in the world around us. Instead, they’ve become locked down walled gardens which force you to use their app or website in order to engage with what’s going on.

Technology should come to us, on our own terms and via whatever medium we choose. This makes projects like RSS-Bridge, rss2email and the myriad of RSS readers out there incredibly important for those who refuse to be locked inside the gardens, but still require access to the information contained within.

If you liked this post and want to see more, please consider subscribing to the mailing list (below) or the RSS feed. You can also follow me on Twitter. If you want to show your appreciation, feel free to buy me a coffee.

Automating routine tasks…

I’ve been thinking more recently about automating routine computing tasks. Using Free Software this is ridiculously easy, usually a combination of Python and Cron does the trick. More difficult I have found is actually working out what to automate. It’s very easy to get into the habit of doing something and then not realise that you do it every day, or every hour when in front of the computer.

One of my major successes in this area has been in terms of feed reading. A while ago I setup rss2email on my server to check my feeds and email the stories to me. Previous to this I actually found it really hard to ever get around to reading news feeds. I think this was mainly because checking news feeds suffers from the ‘extra inbox problem’, its just more stuff to check. Having news stories emailed to me fixes this and I can also read them easily on my phone. Because I use IMAP email, my read/unread status is also synchronised on all my computers.

Now I’m kind of at a brick wall. I don’t know what else I can set up to reduce the number of routine tasks I do. The purpose of this post is to crowd source that problem. I want everyone reading this to get back to me with the things they’ve set up to help them. You can get in touch either through the comments on this post, or via identi.ca. If I get enough responses I’ll write a follow up post detailing some of the best ideas.