bloovis.com

I currently use fetchmail to download my Gmail email via POP3 to my laptop, where I read it with a text-mode email client called sup. But for a couple of months back in 2009, I was in the middle of a cross-country move and temporarily switched to using the Gmail web UI. Then when I switched back to using fetchmail, it wasn’t able to read those two months’ worth of email, which remained missing until today.

To get back the missing messages, I first used the Gmail web UI to label those messages. I knew that I was missing email that had arrived between July 9, 2009 and Aug 29, 2009. So I did a search in Gmail for after:2009/7/9 before:8/29/2009. Then I clicked on the “select all” checkbox, and when Gmail asked me if I wanted to select all 1700 messages in the search results, I said yes. Then I tagged those messages with the “missing” label.

Back on the laptop, I installed mutt, another text-mode email client that I used for many years, and which has excellent IMAP support. I created a ~/.muttrc file that looked like this:

mbox_type=Maildir
set from = "myusername@gmail.com"
set realname = "My Name"
set imap_user = "myusername@gmail.com"
set imap_pass = "mypassword"
set folder = "imaps://imap.gmail.com:993"
set spoolfile = "+INBOX"
set postponed ="+[Gmail]/Drafts"
set trash = "imaps://imap.gmail.com/[Gmail]/Trash"


I created a maildir to store the missing messages:

maildirmake ~/Maildir/missing


Finally, I started mutt. It showed my Gmail inbox, so I told it to switch to the “missing” pseudo-folder by typing c=missing. I tagged all the messages in that folder using T., then saved them all to the newly created Maildir folder using ;s~/Maildir/missing. It took several minutes to download many megabytes of email. After that was done, I now had a fully-populated mail directory containing all the missing messages.

Each time I install Linux Mint on a new machine, or upgrade an installation, I have to perform a set of tweaks to get the system to look and behave the way I want. After doing these tweaks yet again over the weekend, I decided to write them down for posterity.

• Before IBM introduced the PS/2 in the mid-80s, the Control key on terminals and PCs was always where God intended it: next to the A key. The PS/2 keyboard moved the Control key to a very inconvenient location on the bottom left corner of the keyboard. All other PC makers copied this incredible blunder. Finger tendons have suffered ever since, especially those of Emacs users. To fix this: Menu / Control Center / Assistive Technologies / Keyboard Accessibility / Layouts / Options / Ctrl key position / Swap Ctrl and CapsLock.
• Make bash the default shell: sudo dpkg-reconfigure dash, answer No.
• The compiz window manager provides useless eye candy and doesn’t honor some window manipulations, like roll-up. To disable it: Menu / Control Center / Appearance / Visual Effects / None. This must be done before changing window behavior.
• I like seeing all windows in the task bar when I switch workspaces. To enable this, open Window List Preferences, and enable Show windows from all workspaces and Restore to native workspace.
• Enable Emacs editing keys in Firefox: gconftool-2 --set /desktop/gnome/interface/gtk_key_theme Emacs --type string
• Install useful Firefox addons: Adblock Plus, Keyconfig, Flashblock, LastPass, ViewSourceWith.
• I dislike having to enter my password when the screensaver comes on after an idle period. To disable this behavior: Menu / Control Center / Screensaver / uncheck Lock screen when screensaver is active.
• Add the volume control to panel: right-click on panel, select Add to Panel, choose Indicator Applet.
• Install the SSH server to allow remote shell access: sudo apt-get install openssh-server
• To enable two-finger scrolling on the touchpad: go to Menu / Preferences / Mouse / Touchpad, where the option can be found.
• Suspend when the lid is closed: Menu / Preferences / Power management / When laptop lid is closed: Suspend.
• My editor has bindings for F1 and F11, but gnome-terminal uses these as shortcuts for Help and Full Screen, respectively. This page has the solution. To disable F1 in gnome-terminal: Edit / Keyboard Shortcuts. Select action Contents, click on F1, hit Backspace to change the binding to Disabled. Do the same for F11.
• Install a bunch of useful packages: keepassx, openssh-server, gnome-ppp, dvdbackup, flac, vorbis-tools, ruby, rubygems1.8, ruby-dev, libtcltk-ruby, cdparanoia, tkdiff, sun-java6-bin, sun-java6-jdk, sun-java6-plugin, sun-java6-jre.

For an upcoming plane trip, I wanted to extract one chapter from an Intel IA32 Architecture Software Developer manual for reading on my Kindle. In the past, I would have used pdftk for this purpose, but this is an old, unsupported tool that cannot handle the AESV2 encryption used in Intel’s manuals. I then tried to use the Multivalent tools, which supposedly contain various PDF manipulation tools, but this package seems to have suffered from software rot, and the supplied .jar file no longer contains the necessary classes.

Finally, I stumbled on this article in Linux Journal, which shows how to use gs (Ghostscript) to extract pages. This works well enough with Intel’s manuals, though it does change various Microsoft fonts to their more standard equivalents. Here’s a slightly modified version of the article’s script:

#!/bin/sh

# this script take 3 arguments:
#     $1 is the first page of the range to extract
#     $2 is the last page of the range to extract
#     $3 is the input file
#     output file will be named "inputfile_pXX-pYY.pdf"

first=$1
last=$2
input=$3
if [ -z "$first" -o -z "$last" -o -z "$input" ] ; then
   echo "usage: pdfextract firstpage lastpage inputpdffile"
   exit 1
fi
gs -sDEVICE=pdfwrite -dNOPAUSE -dBATCH -dSAFER \
       -dFirstPage=$first \
       -dLastPage=$last \
       -sOutputFile="${input%.pdf}_p$first-p$last.pdf" \
       $input

The new Amazon Kindle app for Android stores its books in the “kindle” directory on the phone’s SD card. I naively assumed that I could copy any non-Amazon but Kindle-compatible books into that directory and have the app recognize them. I tried this with a mobipocket Jane Austen collection (a .prc file) that works just fine on the Kindle, but the Android app crashed immediately after display the book’s title and author.

The trick to getting such a book to be recognized on Android is to use Amazon’s free personal documents service to convert the file to a DRM-ified .azw file.

First, I emailed the .prc file as an attachment to my free personal documents email address: EXAMPLE@free.kindle.com (obviously, you must replace EXAMPLE with your configured Kindle email address). Within a few minutes Amazon converted the file to a .azw (DRM-ified version of the original file), and replied with an email that included a download link for the converted file.

I saved the file to a temporary directory on my computer and renamed it by changing the .azw extension back to .prc. This renaming is very important; otherwise, the Android app won’t recognize the file.

Finally, I copied the file to the kindle directory on the phone’s SD card. The Kindle app can see the file and display its contents.

[Note: I tested this procedure on a Nexus One; no guarantees about other devices.]

It looks like the upgrade to the latest Ubuntu is going to keep me busy solving problems for a while.

Today’s second problem has to do with the fingerprint reader in the ThinkPad X41. There’s a good source of information here, but there wasn’t a definitive set of instructions for Ubuntu 10.04 that actually worked. Most of the uproar about the fingerprint reader in Ubuntu 10.04 has to do with a bug where the Enter key has to be pressed after swiping your finger. I couldn’t even get to that point; the trouble was getting logins to prompt for a finger swipe.

As per the instructions at ThinkWiki, I installed and configured the required packages from the standard repository (no PPAs):

sudo apt-get install thinkfinger-tools libpam-thinkfinger
sudo /usr/lib/pam-thinkfinger/pam-thinkfinger-enable

Then I was able to use tf-tool --acquire and tf-tool --verify to show that the fingerprint device worked. But I was not able to use tf-tool --add-user USERNAME to create a fingerprint file for use by the authentication system; this build of tf-tool did not support that option. So I had to set things up manually, by acquiring the fingerprint file and placing it in the proper directory, with the proper name, and with the proper permissions:

sudo su    # login as root
cd /etc/pam_thinkfinger
tf-tool --acquire USERNAME.bir
chown USERNAME:root USERNAME.bir
chmod 400 USERNAME.bir

In all of these commands, substitute your ordinary user name for USERNAME. After this is done, authentication prompts, either in a terminal (e.g., with sudo) or in X (e.g., the login screen), should ask for either a password or a finger swipe. Due the known aforementioned bug, it may also be necessary to hit Enter after the swipe.

I have an old Logitech QuickCam for Notebooks that uses the pwc (Philips Web Cam) driver on Linux. This camera has always worked flawlessly on Linux Mint 6 (Ubuntu 8.10). But on Linux Mint 9, the camera only worked the first time it was plugged in; on subsequent plug-ins, no programs could read images from the camera. The fswebcam utility reported a timeout trying to read the frame buffer; other programs like Cheese or Skype simply displayed blank images.

The solution is to unplug the camera (if it is not already unplugged), then forcibly remove the pwc driver:

sudo modprobe -r pwc

Then the next time the camera is plugged in, the pwc driver will be loaded automatically and will work properly. This has to be done every time you unplug the camera. I’m not sure why this is necessary with recent Ubuntu releases.

My employer’s VPN system doesn’t allow more than one login at a time. But there are occasions when I’d like to be able to use the VPN from two different laptops simultaneously. The solution, most of which I found here, is to use iptables on the machine running the VPN to forward packets from the machine not running the VPN.

In my case, the Juniper VPN software (ncsvc) sets up a connection on the net device tun0, and the network address is 10.0.0.0. So after I start the VPN on one machine, I run the following script on that machine:

#!/bin/sh
# Share the VPN connection with other machines on the local net.
# The assumption here is the the VPN network is 10.0.0.0.
if [ `id -u` -ne 0 ] ; then
   echo "You are not root.  Rerunning with sudo..."
   sudo $0
else
   echo "1" > /proc/sys/net/ipv4/ip_forward
   iptables -A FORWARD -i eth0 -d 10.0.0.0/8 -j ACCEPT
   iptables -t nat -A POSTROUTING -o tun0 -j MASQUERADE
   sysctl net.netfilter.nf_conntrack_acct=1
fi

Be sure to replace the network address (10.0.0.0), VPN net device (tun0) and local net device (eth0) with the correct values for your system.

Then on the machine that is not running the VPN, I run the following script:

#!/bin/sh
sudo route add -net 10.0.0.0 netmask 255.0.0.0 gw VPNHOST
sudo cp /etc/resolv.conf.vpn /etc/resolv.conf

In this script, replace VPNHOST with the hostname of the machine that is running the VPN (i.e., the name of the machine that is running the first script above). I use static IP addresses on all of my machines, and have added entries for these addresses to /etc/hosts on all machines. I’m not sure how this would work with dynamic IP addresses (DHCP).

The last line of this script is the one new thing I’m doing differently from the scripts at the aforementioned link. It makes the non-VPN machine’s name resolution configuration file identical to that of the VPN machine. This allows the non-VPN machine to resolve hostnames residing on the VPN. In order for this to work, I had earlier copied /etc/resolv.conf from the machine running the VPN to the non-VPN machine, and renamed it to /etc/resolv.conf.vpn.

I just spent many hours getting a 4GB USB key into a state where it can be used to boot Linux Mint 9 (based on Ubuntu 10.04). Here are some notes on the problems I had to solve.

In the past, I’ve used unetbootin to create bootable USB keys. When I did this with the Linux Mint 9 ISO image, the resulting USB key booted up just fine, but the installer application hung when trying to run the partition editor. I narrowed this down to an assertion failure in libparted. Apparently it doesn’t like the geometry on some USB keys. This is a known bug.

Fortunately, the ThinkPad R61 has a CD drive, so I created a CD from the ISO image and booted that. Then after installing Mint 9 on the hard disk and booting that, I was able to create a bootable USB key that didn’t crap out in the installer. I did this by using Mint 9’s built-in Startup Disk Creator, which is in the main Menu under Administration. I told it to erase the entire USB key, which apparently reconfigured its partition table into a form that libparted liked. I verified this by rebooting the R61 with the USB key and running gparted.

The next set of problems came when I tried booting an older ThinkPad R50p with the USB key. As in all recent ThinkPads, you can choose the boot device by pressing the blue ThinkVantage or Access IBM button at startup, then pressing F12. But on the R50p, the USB key wasn’t shown as a boot device. I ran Setup (IBM’s term for BIOS configuration), and in the Startup screen I tried to enable the USB HDD device. This didn’t work; the machine just beeped at me. After a long bout of head-scratching, I finally figured out that Setup doesn’t allow more than eight boot devices to be enabled. I disabled a couple of useless boot devices (e.g., HDD1 and USB Floppy), and then I could enable the USB key.

Finally the USB key would start to boot on the R50p. But after the Linux Mint logo came up, the machine hung with the hard disk light on solid. I rebooted from the USB key and selected the recovery mode boot option in Grub (I forget the exact name for it). Then I was able to see the last error message at the hang. It was an I/O error accessing the fd0 device, which is the floppy. More headscratching and Google searching revealed that I needed to disable the floppy device in Setup.

Now I was able to get past the hang on booting from the USB key, but then X refused to run, displaying a dialog box saying that it was going to run in low graphics mode. This turned out to be a problem with the kernel mode setting (KMS) feature of recent Ubuntu releases. Apparently this is a very bleeding-edge method of determining graphics card type, and it doesn’t appear to work on older machines. The fix was to provide the nomodeset kernel parameter when booting. I did this by hitting Enter when the Grub menu came up, then hitting Enter again to edit the kernel command line. I add the nomodeset parameter and pressed Enter a couple of more times to boot. This time the system was able to boot to a good desktop, though the Mint logo didn’t appear during the early boot stages.

I upgraded one of my ThinkPads from Linux Mint 7 to Linux Mint 9 today, and discovered that I was no longer able to use my HP P1006 printer. The HP printer tool (hp-toolbox) detected the printer correctly, downloaded the appropriate plugin, but then couldn’t install the plugin. Running the tool from the command line didn’t give any extra information. This is a known bug in Ubuntu 10.04.

The solution is to download and build the HP printer package (known as hplip) from source. The complete instructions are here. I had to install the following extra packages before the configure and make steps would succeed: libjpeg-dev, libsnmp-dev, libcups2-dev, libusb-dev, python-dev, libsane-dev, and libcupsimage2-dev.

Keyring is a fine little open source application for Palm OS that stores and generates passwords. There is no Android version of this program, so I decided to move to KeePass 1.x, both the Linux version and the Android version.

Then I hunted for way to import my Keyring data into KeePass 1.x. I couldn’t find a tool to do this, so I had to invent my own method, which included writing a conversion script in Ruby.

The first step is to generate an XML version of the Keyring data file, using this Java program. I fed it the Keyring data, a file called Keys-Gtkr.pdb that I had previously backed up onto my Linux machine using pilot-xfer. It produced the XML equivalent, which I redirected to a file.

Then I wrote a Ruby script called kr2kp, which reads the Keyring XML file and outputs KeePass-compatible XML. I saved the output of this script to another file.

The Android version of KeyPass doesn’t import XML, so I turned to the Linux version, and used it to import the XML file I had created in the previous step. Then I saved the resulting database into a file called keyring.kdb.

Next, I copied keyring.kdb to the keepass directory on the Android device’s SD card. (The exact path of the SD card will depend on the directory where your computer mounts the SD card when you connect the Android device via USB.)

Finally, I ran the Android KeePass app and opened the keyring.kdb file. The Android app is somewhat limited, and doesn’t let you move entries between groups or rename groups. So that kind of manipulation has to be done on the Linux side, and then the .kdb file has to be copied to the Android SD card, as described above.