Deleting thumbnails for inexisting photos

Freedesktop for some years already has a spec on how applications should manage image thumbnails (use Next link there). The spec is now followed by majority of Gnome and KDE applications, including F-Spot, which is one of the very few applications that uses large 256x256 thumbnails under ~/.thumbails/large.

The spec specifies to store thumbnails in PNG format, naming the files after the MD5 sum of the original URLs of the original files, eg 81347ce6c37f75513c5e517e5b1895b8.png.

The problem with the spec is that if you delete or move image files, thumbnails stay there and take space (for my 20000+ photos I have 1.4Gb of large thumbails).

Fortunately, you can from time to time clean them by using simple command-line tricks, as the original URLs are stored inside of thumbnail files as Thumb:URI attributes. I don't recommend erasing all of your thumbnails, because regeneration will take time.

In order to create a list of matching thumbnail-original URL pairs, you can run the following in a terminal inside of either .thumbnails/large or .thumbnails/normal directories (it will take some time):

for i in `ls *.png`; do
identify -verbose "$i" | \
fgrep Thumb::URI | sed "s@.*Thumb::URI:@$i@" >> uris.txt;
done

This will get you a uris.txt file, where each line looks like the following:

f78c63184b17981fddce24741c7ebd06.png file:///home/user/Photos/2009/IMG_5887.CR2

Note that the provided thumbnail filenames (first tokens) can also be generated the following way from the URLs (second tokens) using MD5 hashes:

echo -n file:///home/user/Photos/2009/IMG_5887.CR2 | md5sum

After you have your uris.txt file, it can be easily processed with any familiar command-line tools, like grep, sed, awk, etc.

For example, in order to delete all thumbnails matching 'Africa', use the following:

for i in `cat uris.txt | fgrep Africa`; do rm $i >/dev/null; done

So, as you can see, it is pretty simple to free a few hundred megabytes (depending on the number of thumbnails you are deleting).

With this kind of trick you can even rename the thumbnails of moved files if you use md5sum to generate the new filenames from the URLs, as shown above. This will save you regeneration time.

Versioning your home directory or documents with Git

Git is a relatively new Version Control System, initially started by Linus Torvalds in order to manage the source code of Linux Kernel.

Although Randal Schwartz has stated that Git was not designed to version your home directory, it seems that many people are now trying to do so :-)

Some people have used CVS or Subversion for this purpose in the past, but to my mind, Git is suited better for this task for several reasons:

• Git is grep-friendly (only stores it's metadata in a single .git directory at the root of working copy)
• It is very easy to work with a local repository (just do git init and you're ready)
• Git stores changes very efficiently (even binary files), so not much disk space is wasted, but don't forget to call git gc from time to time
• Git repository is always available on your computer, even when you are offline, but on the other hand it is very easy to push your changes to a remote repository as well
  

All these things are much worse with CVS, which spams all versioned directories with CVS subdirs and stores each version of binary files fully. Subversion also requires more effort to setup, is less storage-efficient, and puts .svn subdirs everywhere.

Having said that, my setup is ultra-simple compared to others on the net!

To start versioning your home directory, just run this in the root of your home:

git init

This will initialize an empty local Git repository in ~/.git/ - this is the location that you can use when doing backups, but otherwise you shouldn't care about it anymore.

Then you need to tell Git to track your important files:

git add Documents
git add bin
git add whatever else you want to version
git commit -m "Adding initial files"

Then you can work normally with your tracked files and occasionally commit your changes to the repository with

git commit -a "description of changes you have done"

Note the "-a" above, that means to commit any changes made to any previously tracked files, so you don't have to use git add again. But don't forget to git add any new files you create before committing.

Use git status to show what files were changed since your last commit. Unfortunately, it will also list all untracked files in your home directory, so you may need to create a .gitignore file. You can get the initial version of this file using this command:

git status | awk '/#/ {sub("/$", ""); print $2}' > .gitignore

then, edit it and possible replace some full names partly with '*'. Don't forget to git add and git commit this file as well!

That's, basically, it! You may also try some GUI tools provided by git, eg gitk or git gui to browse your changes and do some changes if you can't remember the commands.

Moreover, I have some more ideas how to make all this more automatic that I am going to try laster:

• Put git commit -a to user's crontab in order to commit changes automatically, eg daily
• Create a couple of nautlus scripts (located in ~/.gnome2/nautilus-scripts) to make adding, comitting and other actions available directly from Nautlilus file manager in Gnome.

Happy versioning! And read the Git tutorial with either man gittutorial or on the official site.

Use LightZone from F-Spot as external editor

In a previous post I have shown how to teach LightZone, a non-destructive photo editor, to open files passed from the command-line (for some strange reason it doesn't do it out-of-the-box).

Now, I want to be able to use LightZone as an external editor from F-Spot, which I use for my photo workflow.

F-Spot has a convenient Open With menu when right-clicking on a photo, we just need to add LightZone to this menu. After some research, it appear that F-Spot uses the standard desktop entry specification files to populate this menu. These files can be located either in /usr/share/applications or in user's home, ~/.local/share/applications. You can use either location, but I prefer the latter one, because I have unpacked LightZone into my home as well.

Here is the working lightzone.desktop file:

[Desktop Entry]
Version=1.0
Type=Application
Name=LightZone Photo Editor
Exec=LightZone %u
TryExec=LightZone
Icon=/home/USERNAME/LightZone/LightZone_32.png
Terminal=false
Categories=Graphics;2DGraphics;Photography;RasterGraphics;GTK;
StartupNotify=true
MimeType=image/tiff;image/jpeg;image/x-canon-cr2;image/x-canon-crw;image/x-nikon-nef;image/x-pentax-pef;

This file expects that you have followed the previous post and already have LightZone in the PATH which accepts a filename to open on the command-line.

• Exec - this line specifies what command to run, %u means to pass the selected file's URL on the command-line. For some reason with trial and error, I found that F-Spot is only able to pass URLs like this, specifying %f doesn't work with F-Spot. But if you look at the LightZoneOpener.java source, you will see that it supports URLs as well as filenames.
• Icon - change this one to the full path of LightZone's icon, it supplied in the original archive.
• Categories - this specifies where LightZone will appear in the ''Applications'' menu.
• MimeType - here you must list all image mime types that you want LightZone to open. This is especially important for RAW files. As I own a Canon camera, I have most of my photos in CR2 format, so I need to be sure that image/x-canon-cr2 is in the list. I have also specified CRW, NEF and PEF mime types for older Canon, Nikon and Pentax cameras, respectively. These mime types are already registered in Ubuntu Intrepid (not sure about other distributions). Here is some info on how to register new mime types in Gnome, in case your camera's format is not registered yet.
  

After chosing the Open With->LightZone, F-Spot will ask whether to create a new version for the file. Select 'No' - this won't work with RAW files and LightZone anyway, because LightZone saves files automatically with the _lzn.jpg suffix and F-Spot doesn't know about it.

Getting this right requires patching F-Spot (I am going to do that later). For now, you will have to import the saved file manually, if you want it to appear in F-Spot.

Opening files with LightZone from command-line

LightZone is a very useful commercial photo editor with some unique features like non-destructive and layer-based editing. To my mind, the developers has taken a very clever approach to save resulting edits inside of smaller-size JPEG files (thumbnails), so that any program can be used for previewing the resulting image, but opening of the file in LightZone will load the original image and show all the edits again with the possibility to make any changes and export a full-resolution image. The cool thing here is that edited files are very small especially when compared to 10Mb+ source RAW photos, contain all the editing history and can be previewed quickly with any software. And all this runs on Linux (thanks to Java - write once, run almost anywhere).

The only problem with LightZone (at least the Linux version) is that it doesn't accept filenames from the command-line! You have to start the program and select the file manually using the embedded file browser. Of course, this is not an option if you want to run LightZone from another application as an external editor, eg from F-Spot (more on this in a later post).

To make a long story short, I have written a small Java program that takes a filename on the command-line and then modifies LightZone preferences, so when you run it next time it will directly open the specified image.

Here it is:

import java.io.*;
import java.net.*;
import java.util.prefs.*;

/**
 * LightZoneOpener - will modify LightZone preferences to open the 
 * specified file (image) on next startup.
 * This is useful to force LightZone to open a particular file from 
 * the command-line, just run this code before starting LightZone. 
 *
 * @author Anton Keks
 */
public class LightZoneOpener {

  public static void main(String[] args) throws Exception {
    if (args.length != 1) {
      System.err.println("Please specify filename to open in LightZone");
      System.exit(1);
    }
    String filename = args[0];
    if (filename.startsWith("file:"))
      filename = new URI(filename).getPath();
    File file = new File(filename).getCanonicalFile();
    if (!file.exists()) {
      System.err.println(file + " doesn't exist!");
      System.exit(2);
    }
    File fileDir = file.getParentFile();
    
    // set image folder as current one
    Preferences folderPrefs = Preferences.userRoot().node("com/lightcrafts/ui/browser/folders");
    int i = 0; File dir = file;
    while ((dir = dir.getParentFile()) != null) {
      folderPrefs.put("BrowserTreePath" + i++, dir.getName().isEmpty() ? "/" : dir.getName());
    }
    folderPrefs.remove("BrowserTreePath" + i);
    
    // set selected image in the current folder
    Preferences appPrefs = Preferences.userRoot().node("com/lightcrafts/app");
    appPrefs.put("BrowserSelectionMemory" + fileDir.getPath().hashCode(), file.getPath());
    // tell LightZone that last startup was OK just in case
    appPrefs.put("StartupSuccessful", "true");
    
    System.out.println("LightZone is now ready to open " + file + " on next start");
  }
}

Here is what to do:

1. save it to LightZoneOpener.java
   
2. compile with javac LightZoneOpener.java
3. run with java LightZoneOpener

Then you can create a small script that will automate the stuff for you (save it to ~/bin/LightZone):

#!/bin/bash
java -cp ~/bin LightZoneOpener "$@"
~/LightZone/LightZone

This assumes that you have extracted LightZone to ~/LightZone (in your home dir) and have the following two files in the ~/bin dir: the compiled LightZoneOpener.class and the script file LightZone (don't forget to set the execute permission with chmod a+x ~/bin/LightZone)

Here is all this pre-compiled. Just extract the file directly in your home and it will put all needed files into the bin directory. After next login your local bin will be in $PATH, so you will be able to use it.

Now you can run LightZone filename on the command-line in Linux (or using Alt+F2)! Have fun!