Blogging to Nowhere » Tips

Using Android WebViews

Rob Connolly — Wed, 09 Mar 2011 19:00:33 +0000

Recently I’ve been working on show notes support for SwallowCatcher. Since most podcast feeds include their show notes in the feed as embedded HTML I decided to render this HTML to display the show notes. This turned out to be ridiculously simple thanks to Android’s WebView class. The API for rendering arbitrary HTML within your app is almost Pythonic in its simplicity. However, there are a couple of gotchas which caught me out, so I decided to cover them here.

I started out by creating a new Activity class and a layout XML file for it. The layout XML was taken almost directly from the Android WebView Tutorial page:

 version="1.0" encoding="utf-8"?>
  xmlns:android="http://schemas.android.com/apk/res/android"
    android:id="@+id/shownotes"
    android:layout_width="fill_parent"
    android:layout_height="fill_parent"
/>

In my Activity class the XML is loaded as normal. We also need to import the WebView class and the URLEncoder class. Additionally, the UnsupportedEncodingException class is required:

import android.webkit.WebView;
import java.net.URLEncoder;
import java.io.UnsupportedEncodingException;

Next, we need to encode the text in the correct format. Weirdly, WebView requires that the text is URL escaped, but without spaces converted to ‘+’ symbols. I used URLEncoder to encode my content then replaced the ‘+’ symbols with spaces. This probably isn’t perfect, but for my purposes it works.

String text = "Content goes here!";
text = URLEncoder.encode(text, "utf-8");
text = text.replace('+', ' ');

Finally, we find the WebView from the XML and tell it to load the string as its content:

WebView web = (WebView)findViewById(R.id.shownotes);
web.loadData(text, "text/html", "utf-8");

All the above is encased in a try..catch statement for UnsupportedEncodingException, just in case the URLEncoder has a fit. That’s pretty much it. In SwallowCatcher the ShowNotesActivity is 56 lines, including all the verbose Java imports and bootstrapping and loading the content from the database via ORMLite.

As a wise action hero (and ex-Jedi) once said, “I love it when a plan comes together”.

Quick QR Code Generation

Rob Connolly — Tue, 08 Mar 2011 22:58:22 +0000

QR Codes are a really easy way to share information between your desktop or laptop and a smartphone, which don’t have the privacy issues inherent in Google’s Chrome To Phone. The excellent FOSS Barcode Scanner app for Android makes scanning these codes and opening the scanned data a breeze. However, generating them on the desktop can be a bit of a pain. There are several websites, browser addons and APIs for generating these codes, but up until now I haven’t found anything really quick and easy.

That is until today, when I discovered the ‘qrencode’ utility. Qrencode does exactly what it says on the tin. It takes some text, encodes it as a QR Code and writes the result as a PNG file. It’s a simple command line tool, which opens up awesome scripting possibilities.

My use case is simply to send the URL of a web page that I’m viewing on my desktop to my phone, via QR Code. I found that qrencode was quite happy to write its output to stdout, which meant I could display the result directly using ImageMagick’s ‘display’ command, e.g:

$ qrencode http://blog.webworxshop.com -o - | display

You’ll notice that by default the QR Code is quite small. We can fix this by increasing the block size:

$ qrencode http://blog.webworxshop.com -s 10 -o - | display

Next, I wanted to automatically pull the text to encode from somewhere to save typing – the clipboard was an ideal candidate. Enter ‘xclip’. Xclip is a command line utility to read and write from/to the X system’s built in clipboard. I used bash’s backtick command substitution to grab text from the clipboard and encode it:

$ qrencode `xclip -o` -s 10 -o - | display

And there you have it, a simple one line command to generate and display a QR Code from the contents of the clipboard. I created a bash script containing the command and assigned it to a keyboard shortcut in Gnome (Ctrl-Shift-Q), so that sharing URLs to my phone is as simple as selecting the text in the location bar and hitting Ctrl-C followed by Ctrl-Shift-Q.

If you want to give this a try, you’ll need to install the utilities discussed, in Fedora these can be installed with:

$ sudo yum install qrencode xclip ImageMagick

Enjoy!

Unofficial Python Module of the Week #2: configobj

Rob Connolly — Mon, 07 Feb 2011 02:23:07 +0000

Welcome to our third instalment of interesting Python modules. Unfortunately I’m a bit late with this section this week – in fact its next week already! The fourth instalment should be along towards the end of the week thus catching me up.

Today we’re going to cover something which isn’t in the standard library, but is nonetheless very useful. The module is configobj which is used for reading from and writing to INI style configuration files files. A simple INI file is shown below:

item1 = value
item2 = value2
 
[ section1 ]
item1 = value
 
[[ subsection ]]
item1 = value

In the above we can see the simple use of items, values sections and subsection. Subsections can be nested down as far a you want, but I don’t think most applications will need many more than two or three levels.

Installation

As this module isn’t in the standard library, we need to install it. On most Linux distros it should be in the package repositories, for example on Fedora 14:

$ sudo yum install python-configobj

Windows and Mac users can install from PyPi by following the instructions on the homepage.

Basic Usage

Reading from a configuration file with configobj couldn’t really be any simpler:

import configobj
config = configob.ConfigObj(filename)
myoption = config['item1']
mysectionoption = config['section1']['item1']
mysubsectionoption = config['section1']['subsection']['item1']

Basically, all you need to do is open a ConfigObj object by passing it a filename, then you just read from it as if its a dictionary object. Sections and subsections appear as nested dictionaries. Writing to the file is just as simple:

import configobj
config = configob.ConfigObj(filename)
config['newoption'] = 'new stuff'
config.write()

No surprises here, you just write to it as if it were a dictionary. All you have to do it call the write() method when you’ve finished, in order to sync everything to disk.

That’s pretty much it for basic usage. There is much more you can do with configobj, including advanced stuff like validation of configuration files. Check out the documentation for more info.

Unofficial Python Module of the Week #1: shelve

Rob Connolly — Thu, 27 Jan 2011 21:06:17 +0000

Here we are, the second Unofficial Python Module of the Week. Yes, the second – we started from zero (obviously!). This week we are covering the shelve module. Shelve provides you with a very simple Python object store. You can use it where you need quick persistent storage of objects between program runs, it’s much less overhead than using a database – even SQLite. Anyway, lets dive straight into it:

>>> import shelve
>>> shelf = shelve.open("myshelf.db", writeback=True)

Here we import the shelve module (its in the standard library, so there’s no installation required). Then we open our persistent object store, supplying the filename that we want to store the objects in and the writeback parameter, which allows mutable objects to be stored more conveniently (otherwise they are only written when an assignment is performed). The writeback parameter also causes data to be cached in memory, which can be quite memory intensive, so you should call shelf.sync() every so on to flush everything to disk.

You can store anything that can be handled by the Python pickle module in a shelf:

>>> shelf['thedict'] = {'one': 1, 'two': 2, 'three': 3}
>>> shelf.sync()

As you can see, using a shelf is just like using a dictionary. The only real limit is that the keys must be strings. You can also read back values from the shelf as with a dictionary:

>>> print(shelf['thedict'])
{'one': 1, 'two': 2, 'three': 3}

That’s just about it, just remember to close the shelf when you’re finished with it:

>>> shelf.close()

If you want to find out more have a look at the official Python docs for shelve and Doug Hellmann’s PyMOTW posting on the subject.

Unofficial Python Module of the Week #0: argparse

Rob Connolly — Wed, 19 Jan 2011 19:00:19 +0000

[EDIT: This series has been renamed so as not to conflict with Doug Hellman's excellent Python Module of the Week series. See UPMotW #1 for details]

I’m going to try something different today. This is going to be the first in a series (hopefully once weekly) of Python Modules of the Week. This has probably been done to death elsewhere, but I’m using it as an opportunity to learn some more Python and to cement some of these modules in my mind.

Argparse is a Python module for processing command line arguments. It’s new in version 2.7 and is designed to replace the Optparse module. I’m not sure what the rationale for the change was as the two modules look very similar to me. However, it suffices to say that you should use argparse in all new code.

The main class of argparse is called ArgumentParser, it allows you to add arguments to your program and parse the provided arguments at runtime:

parser = argparse.ArgumentParser()
parser.add_argument("-v", "--verbose") # a flag type argument
parser.add_argument("value", type=int) # an integer positional argument
...
args = parser.parse_args() # parse the args from sys.argv
print(args.value) # arguments accessed as data members

You can also add meta-data such as help and descriptions to your arguments via keyword arguments to many of the methods. Take a look at the module documentation for examples. Once nice thing is that when you provide this data, argparse automatically generates the -h and –help flags and populates them with nicely formatted help output using the info you provide.

The argument parsing provided by argparse allows you to create some very complex command line interfaces to your scripts. I especially like the sub-commands framework, which allows each sub-command to exist as its own separate entity with its own options and help information. To create an argument parser with a sub-command is just a few lines:

1
2
3

parser = argparse.ArgumentParser()
subcommands = parser.add_subparsers()
mysubcommand = subcommands.add_parser('mysubcommand')

Of course,each sub-command is an ArgumentParser in its own right, so you can add all the options that you normally would, including command line flags, positional arguments and help information. Also each sub-command will be listed in the help and will also have its own help page which can be accessed via:

1	./pyprog.py subcommand --help

All in all, this is a very useful Python module to know about. Its ease of use and the number of things it ‘just takes care of’ make it a pleasure to use. Hopefully it’ll have you writing nice command line interfaces to all those hacked together scripts that are currently controlled by commenting bits of them in and out!

Calculating Alcohol By Volume in Python on Android

Rob Connolly — Wed, 12 Jan 2011 02:14:22 +0000

Wow, I just managed to combine three of my favourite things in a single title! Recently, I’ve been getting further into home brewing, with a book I received as a Christmas present (Home Brewed Beers and Stouts, by C.J.J. Berry). Since I’d never actually measured the Alcohol By Volume (ABV) of my beer I decided to write some Python code to automate the process. The simple modules I came up with work from the command line and also on Android phones via SL4A, which makes them very useful when doing quick measurements.

Measuring ABV involves taking Specific Gravity (SG) measurements at the start and end of fermentation, adjusting them for temperature and pushing them into a simple formula. The SG measurements are taken with a Hydrometer, which is supposed to read at 20C (hence the need to adjust for other temperatures).

The temperature adjustment is done via a simple table (which I took from the book). In the script I used Linear Interpolation, to adapt it for values which weren’t available in the table. Initially I used the numpy.interp() function. However, I found that numpy isn’t present in SL4A and that installing it would be a pain since it is a C module which would need cross compiling for Android.

I therefore wrote my own interp() function, which keeps the same interface:

def interp(x, xp, fp):
    i = 0
    for p in xp:
        if p > x:
            i = xp.index(p) - 1
            break
    return ((x - xp[i])*fp[i+1] + (xp[i+1] - x)*fp[i]) / (xp[i+1] - xp[i])

I split up the code into two separate modules, sg.py (which just does specific gravity adjustment) and abv.py (which does ABV calualation). Splitting the code up enables me to take SG measurements and adjust based on temperature, without doing a full ABV measurement. In sg.py the reverse adjusted SG is what you would need to read from the hydrometer at the specified temperature, in order to achieve the adjusted reading you specified.

Since the calculations are fairly trivial the main bulk of the code is in the user interfaces. I implemented a simple command line interface which either takes arguments from the command line, or prompts the user via the python input() function. I also use the SL4A API to implement a simple UI for Android, basically the user is prompted for each quantity by a dialog box.

Anyway, there’s not much else to say, except that I’ve put the code up on Gitorious for anyone who wants it (it’s licenced AGPL). The code is in a git repository for useful Python scripts I’ve written, right now it’s the only thing there, but I’m going to track down some of the scripts I’ve written over the years and add them too – I might even get a few blog posts out of some of them!

Playing with Python Generators

Rob Connolly — Mon, 30 Aug 2010 08:12:55 +0000

Today I’ve been playing with generators in Python. I have to say they are yet another awesome Python feature! They give you a very cool and efficient way to store some state within a function, this is useful if you’re trying to do something like generating a sequence, which you would otherwise need a class for. I’m going to go over a couple of quick examples here in order to demonstrate what generators are and what you can do with them.

Side note: You need Python 2.4 or above to use generators, you’ll probably have this anyway, but it might pay to check before wondering why this stuff doesn’t work.

A Quick Introduction

At it’s heart a generator is a function which can return a value several times within it’s body. Each time the function is called it resumes from where it terminated in the previous call, with all it’s internal state intact (i.e. variables are held between calls). In Python a generator is defined by creating a function which uses the ‘yield’ keyword. Each time that a yield is hit the function will return the specified value. Let’s start with an example which defines a generator implementation of the built-in ‘range’ function:

def genrange(start, end): i = start while i < end: yield i i += 1

As you can see this is incredibly simple and behaves in the same way as the standard range function. The only difference is that when the generator method is called it will return a generator object rather than a value (in contrast to range, which will return a list of values in the range). These generator objects are iteratable so calling the ‘next’ method on the object will run your code and return the next yielded value. Because the generator object is iteratable, the way in which you are likely to use it should look familiar:

for i in genrange(0, 10): print i

If you want to go ahead and create a list from the sequence as you generate it, you can use a list comprehension, like so:

l = [i for i in genrange(0, 10)]

Of course, as hist comprehensions can be much more complicated than this you can use it to achieve more than just creating the equivalent list as calling the built-in range function. For example, you could filter the output elements according to some criteria, or call a method on each element returned.

A Trade Off

Although the above example is incredibly simple, it presents an interesting trade off between memory use and the speed of iteration through the ‘list’. The trade off exists because you could just build the list of elements in a separate function and then iterate through it. This is a perfectly valid approach but if there are a large number of elements you will use a significant quantity of memory to store the list. You are also more likely to double handle the data, unless you are building the list up over a period of time. Using a generator in this case is likely to be more efficient, as you are just creating the elements as you need them, so only storing a minimal amount of data at any one time.

The other side of the trade off is performance. If the process that produces your iterative data is computationally expensive, it makes no sense to calculate more values than you will need. So if you have a data processing loop, that can potentially exit early (i.e. before reaching the end of the ‘list’) a generator will be more efficient. But equally, if you require a tight loop which iterates quickly, you would want to pre-compute the values.

When considering using generators you should weigh up these factors and think which method is going to be most efficient for your situation. I think in a lot of cases, you will end up coming down in favour of the generator solution.

Another Example (Dan Brown is going to love me)

I’m going to leave you with another quick example, namely a very efficient way of generating Fibonacci numbers using a generator. Without further ado:

def fibonacci(): p1 = 0 p2 = 1 while True: f = p1 + p2 p2 = p1 p1 = f yield f

There you go, easy. Many other Fibonacci implementations use a recursive method, which is horribly inefficient. Most other implementations will require either a class to store persistent data in, or the use of static variables, both of which are more tricky to handle from a programming point of view.

Well that’s it. I think I’ve demonstrated that generators are pretty cool. The examples I’ve presented are very simplistic, but I’m sure you can think of awesome uses for them. Go forth and code!

Denting via Android Intents

Rob Connolly — Sun, 30 May 2010 01:01:36 +0000

This is just a quick post to share something cool that I was playing with the other day. I was wondering if it was possible to send a message from an Android application via the Mustard identi.ca client. It turns out it is, and here’s how you do it:

Intent i = new Intent(Intent.ACTION_SEND); i.setType("text/plain"); i.putExtra(Intent.EXTRA_TEXT, "Testing intents on android"); startActivity(i);

These four lines of code hook into a key feature of the Android platform: Intents. The intent system is a neat way of communicating via apps and enables different apps to integrate together without knowing about each other specifically.

In this case, we are sending the ACTION_SEND intent, which Mustard is set up to handle. We set the mime-type of the message to ‘text/plain’ and put the contents of our message into the EXTRA_TEXT field. We then start the intent, and we’re done!

When the intent is started Android presents the user with a list of all the possible applications which can handle the ACTION_SEND intent. In my case this is Email, Gmail, Messaging and Mustard, when I select mustard the new message screen pops up and is populated with my message content. Hitting send posts the message and returns control to my app. Neat!

There are probably a whole host of ways you can integrate your apps with other apps on the phone using the intents system, I guess it’s just a matter of knowing what intents are received by each app and what they do.

Thanks to @macno (main developer of Mustard) and @bugabundo on identi.ca for their help working this out!

Anyway, I hope someone finds this useful. This is the first time I’ve posted about Android development, but I’ll hopefully write more about it in the future. Bye for now!

Online Filesystem Resizing with LVM

Rob Connolly — Fri, 09 Oct 2009 23:45:27 +0000

I use LVM on my main desktop machine. This is awesome because it allows me to dynamically allocate space to partitions as I choose, however I always forget how to do a resize, so I’m going to write it down here. This isn’t going to be a full LVM tutorial (there’s plenty of material out there for that), although maybe that’s an idea for the future.

The following commands will resize an ext2, ext3, or ext4 filesystem running on LVM while it is mounted:

$ sudo lvresize -L +XXG
$ sudo resize2fs

In the above command you need to replace XX with the number of GB you want the filesystem to grow by and by the device node (typically /dev/mapper/something).

An there you have it, done! Obviously there is a huge amount more you can do with the two tools above, take a look at their man pages for more info.

Hopefully this post will save me from having to work out how to do this every time!

Quickly change Debian repositories

Rob Connolly — Wed, 03 Jun 2009 08:30:30 +0000

Apt is awesome. Plain and simple.

But it is kinda static. By this I mean it’s not particularly suited to environments where things change frequently. For example, we have a local mirror at uni, which of course it much faster than using the external Ubuntu or Debian ones, however as this is only available from internal University of Auckland IP addresses I would have to change my /etc/apt/sources.list file if I wanted to install something from home.

Today I knocked together a quick Python script to fix this, all it does is basically manipulate a symlink which points to the real /etc/apt/sources.list file, but I thought I’d share it anyway:

#!/usr/bin/env python import sys, os def usage(): print "Usage: %s " % (sys.argv[0],) def main(argv): if argv[1] != None: sources_path = "/etc/apt/sources.list.%s" % (argv[1],) if os.path.exists(sources_path): os.unlink("/etc/apt/sources.list") os.symlink(sources_path, "/etc/apt/sources.list") print "SUCCESS: Don't for get to run 'sudo apt-get update'" return 0 else: print "ERROR: Unknown repository set '%s'" % (argv[1],) usage() else: usage() return 1 if __name__ == '__main__': sys.exit(main(sys.argv))

Unfortunately wordpress screwed with my formatting there (which kinda matters in a python script) so here is the file (repo-switch.py). You’ll need to download it and put it somewhere in your $PATH. Then make it execuatable:

chmod +x repo-switch.py

The next step is to set up your lists of repositories. Each list will have a suffix which the script will use to identify it. First I copied my main repositories (for the NZ Ubuntu mirrors):

sudo cp /etc/apt/sources.list /etc/apt/sources.list.nz

Obviously replace ‘nz’ with a suffix of your choice. Then I dropped the list of repos for the local mirror into another file (in my case /etc/apt/sources.list.ece). You can set up as many different sets as you like, but I only need two.

Next step is to practice switching between them:

sudo repo-switch.py nz

…sets apt to use the file /etc/apt/sources.list.nz by changing /etc/apt.sources.list to a symlink pointing to the relevant place. Calling the script again with a different set (e.g. ‘ece’ in my case) will set it up to use one of your other sets.

Obviously, you’ll need to run ‘sudo apt-get update’ each time to make apt work with the new repos, but I added a line to the script to remind you!

Magic!