bloovis.com

03/25/2010 (5:31 am)

The impoverished state of Android Apps

Filed under: android, centro, nexus one, ruby, software, treo ::

One of the reasons why I decided to buy an Android device was to investigate the possibility of writing applications for that platform. After spending a couple of days with Android, it’s pretty clear there are huge voids in the app space waiting to be filled. The transition from Palm OS is going to be painful because the following apps have no equivalent on Android:

  • Adarian Money. This is lovely little financial app that I’ve used for several years to track every penny I spend. The Android checkbook apps do not come close; the best-selling one doesn’t even track expense accounts, let alone support double-entry accounting. Clearly, a lightweight version of GnuCash is needed here.
  • DateBk. I’ve used this program for nearly ten years in its various incarnations, from DateBk+ on a Handspring Visor, to DateBk5 on a Centro. It’s the king of calendar apps and nothing on the iPhone or Android comes close. The Google Calendar app does sync with the web version, but is otherwise very minimal. It also has a serious bug: its ringtone reminders do not work when the screen is turned off, so you’ll miss appointments and meetings constantly. Pimlico has hinted that they might be porting DateBk to other platforms, but who knows when or if Android will ever be supported.
  • ListPro. This is a checklist app on steroids, almost a mini-database. I use this for packing lists, lists of items lent out to other people, notes on things to look up when I get home, to-do lists, etc. The Android apps are the usual mix: either seriously broken or only supporting a tiny subset of ListPro’s features.

If I didn’t have a full-time job, I’d start working on filling these gaps myself. I may do that anyway as a rainy-day weekend hobby. The prospect of writing in Java is not pleasant, so I may start working on the algorithms and data structures (or Models and Controllers, in newspeak) in Ruby, and hope that Duby is usable on Android by the time I need to start thinking about the UI (or View in newspeak).

10/06/2009 (1:42 am)

The Kindle DX and PDF metadata

Filed under: kindle, kindle dx, ruby ::

One of the most common complaints about the Amazon Kindle is its lack of support for “folders”. In Linux terms, this means that the Kindle flattens your “documents” directory tree when it displays the list of your books on its home screen. However, a directory-browsing UI would be much less flexible than a tagging system, because it would require you to impose an arbitrary hierarchy on your documents. Fortunately, there is a workaround for this that implements a kind of pseudo-tagging using annotations. The advantage of this workaround is that it’s performed on the Kindle and doesn’t require a separate computer.

Even without tagging, the Kindle allows your document list to be sorted either by author or title. Then you can jump to authors or titles starting with a particular letter by pressing that letter key and clicking the 5-way controller. I find this shortcut adequate for most purposes.

But PDF documents, which are supported in a limited fashion on the Kindle DX, present special problems. First, the tagging workaround can’t be used because annotations are not supported. Secondly, if the author and title aren’t set correctly in the PDF file’s metadata, the Kindle will fall back to using the filename as the title of the document. Even if the title is set correctly, the Kindle won’t display it in the home screen; it will display the filename instead, and only show the actual title when you highlight the file and then 5-way to the right. And finally, many PDF files, such as sheet music from the IMSLP, will have missing or incorrect metadata. (This is a problem with non-PDF ebooks too, such as those from the Baen free library.)

So in order to sort through large numbers of PDF files on the DX, it’s very important to set the author metadata correctly. The free calibre program supposedly can do this, but I choose not to use it because I wanted to use my own scripts and have more control over my directory structure. So instead, I’m using the pdftk utility to modify the metadata in my PDF files. In particular, I’m setting the Author and Title fields, which appear to be the only ones that the Kindle recognizes. (I had hoped that the Kindle would also recognize Keywords, but that doesn’t appear to be the case.)

However, pdftk is a little clumsy to use in this fashion. It reads and writes metadata in a format that’s not very convenient. You have to prepare a text file that looks like this:

InfoKey: Title
InfoValue: Six Piano Pieces Op. 118
InfoKey: Author
InfoValue: Brahms

Then you pass this text file to pdftk using its update_info subcommand. I found this usage annoying, so I wrote a wrapper script in Ruby called pdfmeta that lets you specify the author and title on a single command line. Using the above example, here’s how I would update the author and title for a piece of sheet music that I downloaded from the IMSLP:

pdfmeta -a Brahms -t "Six Piano Pieces Op. 118" op-118.pdf

There’s one catch with this: If you transfer a PDF file to the Kindle, then discover that you need to change the metadata, you’ll need to delete the old file on the Kindle, then rename the updated file (changing or adding one letter is enough) before copying it to the Kindle. Otherwise, the Kindle will continue to use the old, incorrect metadata; apparently it stores this information in a separate index keyed by filename, and doesn’t delete or update that information when you update the file.

02/24/2009 (8:32 pm)

Upgrading from Rails 2.1 to 2.2

Filed under: rails, ruby ::

I have a small project that I developed at work using Rails 2.1. After installing the latest version of Rails, 2.2.2, I had to do the following to get my project working again:

  • Edit config/environment.rb and change the value of RAILS_GEM_VERSION to ‘2.2.2′.
  • Edit config/environments/development.rb and comment out the config.action_view.cache_template_extensions line.
  • Run rake rails:upgrade. This added a new file, script/dbconsole, and modified the following files:
    • config/boot.rb
    • public/javascripts/controls.js
    • public/javascripts/dragdrop.js
    • public/javascripts/effects.js
    • public/javascripts/prototype.js

11/18/2008 (4:41 pm)

Convert tabs to spaces in one line of Ruby

Filed under: ruby, software ::

Here’s a one-liner Ruby script that converts tabs to spaces:

ruby -pe 'gsub(/([^\t]*)(\t)/) { $1 + " " * (8 - $1.length % 8) }'

I’m in the early stages of rewriting a tiny editor, MicroEMACS, in Ruby, and needed a tab converter. I was sure I’d seen this somewhere before, but couldn’t find it with Google, so rewrote it from scratch.

05/14/2008 (3:52 am)

Installing Rails on SLED

Filed under: ruby, software, suse ::

SUSE Linux Enterprise Desktop 10 SP 1 (or SLED), as installed on the ThinkPad R61, is based on SUSE Linux 10.1. This distro includes a somewhat old version of Ruby on Rails, a popular web development framework. I wanted to use the latest version of Rails, but before I could do that, I needed to build and install the latest stable versions of Ruby and Rubygems (Ruby’s package management system). This wasn’t too difficult, but there were a few non-obvious steps along the way. (All of the steps described here were performed while logged in as the root user.)

I first used the Software Management tool in Yast2 to delete the existing Ruby packages I’d previously installed. Then I downloaded the source for ruby 1.8.6 here. Before building it, I had to unset the RUBYOPT environment variable, which was set to “rubygems” by SUSE. Then I built the basic Ruby interpreter using these commands:

./configure
make
make install

This process didn’t build or install the tk extension, which I use in a couple of my Ruby scripts to build simple GUIs. To build that, I first needed to use the Software Management tool in yast2 to install the tcl-devel and tk-devel packages. Then I built and installed the tk extension using these commands:

cd ext/tk
ruby extconf.rb
make install
cd tkutil
ruby extconf.rb
make install

Then I downloaded the latest version (1.1.1) of Rubygems here, and installed it using this command:

ruby setup.rb

Finally, I was able to install the latest Rails using Rubygems:

gem install rails

This installed all of the packages that Rails depends on, such as ActiveRecord.

02/22/2008 (9:26 pm)

Free music formats

Filed under: ipod, linux, music, ruby, software, treo ::

Playing music directly from CDs is, like, so last millennium. I don’t even own a CD player any more, unless a laptop with a CD drive counts. I do all my listening now via a 4th-generation iPod and a Treo 700p.

The problem is that most popular music file formats, particularly MP3, are encumbered by patents. The owners of these patents require license fees if you use files in these formats for commercial purposes, or make them available for downloading via the internet, or copy them to a physical medium like a CD — essentially, you have to pay protection money for any purpose except private use in the home.

Because the legality of patented file formats is questionable for Linux users, I decided to stop using these formats altogether. Fortunately, there are free formats that work just fine: FLAC (a lossless format), and Ogg Vorbis (a lossy format conceptually similar to MP3). My fourth generation iPod didn’t support these formats, but replacing the iPod’s firmware with Rockbox fixed that problem. The commercial Pocket Tunes software for the Treo also plays Ogg Vorbis files.

So now the interesting technical problem was to convert my CD collection to digital files for use on the computer and my two playback devices. My first two attempts at this were failures because I mistakenly chose to rip the CDs initially to Ogg Vorbis and MP3. This was a mistake because these formats are lossy, and some audio fidelity is lost in the conversion process. I could hear the loss in fidelity when comparing a CD of the Brahms First Symphony with an Ogg Vorbis file created with -q3 (equivalent to MP3 at 128Kbps). The opening bars of this symphony are very thick, and the Vorbis file sounded muddy compared with the CD. Increasing the Ogg Vorbis quality level to -q6 cleared up the muddiness, but I realized that I need to start from scratch with the ripping process.

Now my strategy is to first rip CDs to FLAC files. FLAC is a lossless format, so these files are, in essence, an exact copy of the music on the CD. FLAC is quite bulky, anywhere from three to six times the size of an Ogg Vorbis -q6 file of the same recording. But the FLAC files only need to live on the ThinkPad, not on the iPod or the Treo. When the ThinkPad disk fills up, I back up the FLAC files to an external USB disk, then delete them from the ThinkPad disk. I’m not too worried about the lack of redundancy here because the CDs act as the ultimate backup.

Once I have the FLAC files, I then transcode them to whatever lossy format I need for playback, typically Ogg Vorbis. As mentioned earlier, I use quality level -q6 for this step, because I find it produces results that, to my ears, are nearly indistinguishable from the original CD.

The workflow for these steps (ripping, tagging, transcoding, syncing) was not so easy at first. I was using GUI tools like grip, but as with nearly all GUIs, these tools required a large amount of manual labor: filling out forms, clicking buttons, and so forth. The problem was particularly annoying when I had ripped several albums and wanted to convert them all at once to Ogg Vorbis.

To make the process of ripping and tagging more automatic, I invented an album description file format, and wrote some Ruby scripts that use these “album files”, as I call them. Album files are simple text files that contain the artist, album title, genre, date, and track names. Here’s a hypothetical album file:

Artist=Brahms
Album=Piano Works - Radu Lupu
Genre=Classical
Date=1970
Rhapsody in B minor Op. 79 #1
Rhapsody in G minor Op. 79 #2
Intermezzo in E flat Op. 117 #1
...

The first step in ripping a CD is to create an album file for it. There are several online databases of track information that can be used; I use freedb.org. The script cdmakealbum reads the track data from a CD, queries freedb.org for a matching record, and writes a corresponding album file to standard output. I usually hand-edit the resulting album files to correct mistakes or to suit my aesthetic and organizational preferences.

The second step is to rip the CD to FLAC files and tag them based on the information in the album file. The script ripalbum does this. It keeps the files separate by using a two-level directory hierarchy (artist name, album name).

The third step is to trancode the FLAC files to Ogg Vorbis. The script flac2ogg recursively walks the FLAC directory tree and creates Ogg Vorbis files in a separate Ogg directory. The script is smart enough to skip files that have already been converted.

The final step is to copy the converted Ogg Vorbis files to the iPod. Because the iPod runs Rockbox, it’s a simple matter of using rsync to copy a directory on the laptop to a directory on the iPod. The script syncipod does this, after first running flac2ogg (thus eliminating the need for the separate transcoding step described in the previous paragraph).

While this might seem like an awful amount of work, it’s actually quite fast. The lengthiest part of the process is hand-editing the album files; once that’s done, the scripts can run unattended.

Another advantage of the scripts is that some of them take an optional parameter that tells them to pause when the CPU temperature exceeds a certain level. This prevents my flaky ThinkPad T40 from crashing when it gets too hot (which it can do during the very CPU-intensive transcoding process).