How I fixed my new YN565EX flash

Yesterday I have received my brand new YN565EX flash by Yongnuo from Hong Kong. It took almost a month for it to get to Estonia, but shipping was free.

The flash is very similar in specs to a much more expensive Canon 580EX II, with minor exceptions, e.g. no Hi-Speed sync or ETTL Master mode. However, the flash supports ETTL Slave mode of both Canon and Nikon systems, so this was the primary reason for buying it - use as a powerful (guide 58) slave flash unit in automatic TTL mode. As a bonus it can also be a slave in manual mode as well - overall a better option than buying a 430EX from Canon, which is even slightly more expensive, for the same purpose. The reviews on the net were very good and I was impressed with the build quality, which matches flashes made by Canon.

But all would be cool if it worked... There is much scepticism for buying of Chinese flashes, and there seem to be a reason for that. The flash powered on for the first time and then quickly shut down. It completely ignored any presses of the On button afterwards and soon I noticed that the batteries get very hot while they are inserted into the flash unit. Bad, bad, bad. My option would be to pay for return shipping and wait for 2 more months until the new unit is delivered. Not a sexy option.

So I decided to give it a try to fix it myself. Hot batteries suggested there should be a short circuit somewhere and if it is not a major component, it should be fixable.

The flash is very similar in design to Canon 580EX, so searching for its schematics helped to understand how to disassemble it.

1. Unscrew the 4 screws on the bottom of the flash unit to remove hotshoe connectors (contacts can be easily disconnected once the bottom panel is open)

2. Turn the flash head 90 degrees to reveal 2 more screws at the upper part of the body, one is under the sticker, which looks like a 'warranty void' one, but as there are no such words on it - it felt safe to remove it. I suggest making pictures of the screws to be able to assemble everything later - they are all different.

3. Now was the most difficult part with which I struggled for some time. By now you should be able to remove the front cover of the body, but it feels like there is still something holding it at the top - actually there are no more screws, so just try again and again to gently pull it off. No sliding down, etc is necessary, just pull the front panel off the back panel, the position of the head is not important at this time. After you are successful, disconnect the two connectors and put the front cover away.

At this time I put the batteries back inside several times and used a voltmeter to understand where the voltage drop happens. After some tries I was lucky to see some smoke. This might be scary, but I quickly removed the PCB at the bottom of the flash on which the coil and 1000uF capacitor are mounted on the picture. Be careful with touching the parts - you may get an electric shock even if batteries are removed, this is due to a 300V giant capacitor which is hiding at the base of the flash head.

The cause of the smoke most probably was residual soldering agent on the back of the board - see the picture to the right. It was melted and probably was the reason for the short circuit. I used a small screwdriver to clean it off between the two large contacts. Note that checking for it with a multimeter is difficult because there are capacitors mounted here, so probing for resistance can still show some value until the capacitors are charged (and there is a big one there!). After inserting the batteries the flash turned on! The problem was identified, so I have spent some more time cleaning the contacts by gently scratching and wiping off with alcohol.

Now the flash works perfectly both in the hotshoe and as a slave. There are no problems with balance of light with my Canon 580EX II - at least I haven't noticed any difference so far. The only minor problem that I have noticed is the modelling light (when depth-of-field button is pressed) gets darker by the end of the 1 sec interval it is supposed to be visible, but it may depend on how well the batteries are charged.

Overall, I am pretty happy now. Buy yourself one if you have basic disassembling/electronics skills to protect yourself from minor manufacturing defects. I really hope the QC will improve at Yongnuo.

Reusing Shotwell thumbnails in Nautilus

As I have lots of photos on my machine, thumbnails start to consume considerable amount of space on the disk.

Another problem, is that gnome-raw-thumbnailer isn't enabled in Ubuntu (Natty, Oneiric) by default anymore, so my raw photos don't get thumbnailed in Nautilus. And, if I enable it manually, thumbnails of vertical photos don't show with the correct orientation.

So, I have researched a bit the freedesktop thumbnail spec, gnome thumbnailer spec and how Shotwell stores its thumbnails and came up with a shell script that reuses Shotwell thumbnails for Nautilus.

Save the script below as /usr/bin/shotwell-raw-thumbnailer

#!/bin/bash
input=$1
output=$2

if [ -z $output ]; then
    echo "Usage: $0 input output"
    exit 1
fi

file=`echo -n ${input##file://} | perl -pe 's/%([0-9a-f]{2})/sprintf("%s", pack("H2",$1))/eig'`
md5=`echo -n $input | md5sum | awk '{print $1}'`

shotwell_id=`sqlite3 ~/.shotwell/data/photo.db "select id from PhotoTable where filename = '$file'"`
if [ -z $shotwell_id ]; then
    gnome-raw-thumbnailer $input $output
    exit
fi

thumb=`printf ~/.shotwell/thumbs/thumbs128/thumb%016x.jpg $shotwell_id`
if [ \! -e $thumb ]; then
    gnome-raw-thumbnailer $input $output
    exit
fi

replaceWithLink() {
    sleep 1
    ln -sf $thumb ~/.thumbnails/normal/$md5.png
}

# gnome-thumbnail-factory doesn't support links
cp $thumb $output

# however, linked thumbnails work, so replace them after a delay
replaceWithLink &

In order to make it work, you then need to register it as a thumbnailer in Gnome, put this to /usr/share/thumbnailers/shotwell.thumbnailer

[Thumbnailer Entry]
Exec=/usr/bin/shotwell-raw-thumbnailer %u %o
MimeType=image/x-3fr;image/x-adobe-dng;image/x-arw;image/x-bay;image/x-canon-cr2;image/x-canon-crw;image/x-cap;image/x-cr2;image/x-crw;image/x-dcr;image/x-dcraw;image/x-dcs;image/x-dng;image/x-drf;image/x-eip;image/x-erf;image/x-fff;image/x-fuji-raf;image/x-iiq;image/x-k25;image/x-kdc;image/x-mef;image/x-minolta-mrw;image/x-mos;image/x-mrw;image/x-nef;image/x-nikon-nef;image/x-nrw;image/x-olympus-orf;image/x-orf;image/x-panasonic-raw;image/x-pef;image/x-pentax-pef;image/x-ptx;image/x-pxn;image/x-r3d;image/x-raf;image/x-raw;image/x-rw2;image/x-rwl;image/x-rwz;image/x-sigma-x3f;image/x-sony-arw;image/x-sony-sr2;image/x-sony-srf;image/x-sr2;image/x-srf;image/x-x3f;

So, what does this script do?

• When Gnome (or Nautilus) needs a thumbnail, it runs this script
• The script checks if the image has an entry in the Shotwell database (~/.shotwell/data/photo.db)
• Then it checks if Shotwell has a thumbnail for it (in ~/.shotwell/thumbs)
• If yes, the script returns the already generated thumbnail to Gnome - no generation needed, so it works much faster
• If Shotwell doesn't have the thumbnail, the call is delegated to gnome-raw-thumbnailer that generates a new thumbnail, the old-fashioned way
• If Shotwell's thumbnail was used, the script will asynchronously replace the thumbnail in ~/.thumbnails with the link to Shotwell's file, avoiding a copy on the disk

The last step is the one that saves disk space. Unfortunately, it is not possible to return a link right away to Gnome - it can't read it for some reason. However, by putting a link directly under ~/.thumbnails later works perfectly, even if we put a .jpg file under the name of .png (as required by a spec). Png is actually a worse choice for thumbnailing of photos due to its lossless compression, so the disk savings are more than twofold with this script.

The next step would be to rewrite this in C or Vala to make even faster and maybe even make Shotwell create these links right away when it generates the thumbnails.

Simple Rsync GUI: easy backups from Nautilus

Most often, making backups of your important files is a manual process. Especially if you are dealing with large collections of photos.

In the meantime I have written a small and convenient Nautilus script (for Gnome users) for doing exactly that.

Features:

• Syncs to any mounted location or over SSH (everything that rsync supports)
• Remembers previously used locations
• Preview of changes (any deletions are shown first, but performed the last)
• Nice progress bar with upload speed display

Everything is written as a simple bash script using Zenity for GTK GUI - just drop it to ~/.gnome2/nautilus-scripts directory, and it appear in Nautilus right-click menu, under Scripts.

Don't forget - this all is just a frontend for rsync (that you are too lazy to run from command-line).

Dependencies: nautilus, zenity, rsync, bash

And now, here is the source (save to ~/.gnome2/nautilus-scripts/Sync):

#!/bin/bash
# Nautilus script to sync specified folder to another destination via rsync.
# Put this to ~/.gnome2/nautilus-scripts
# Written by Anton Keks (BSD license)

paths_file=$(readlink -f $0).paths
locations=`cat $paths_file`
sources=`cat $paths_file | awk -F'|' '{print $1}'`

if [ "$1" ]; then
  source=$1 
else
  # add current directory also to the list
  sources=`echo -e "$sources\\n$PWD" | sort -u`
  # ask user to chose one of the sources
  source=`zenity --list --title="Sync source" --text="No source was specified. Please choose what do you want to sync" --column=Source "$sources" Other...` || exit 1
  if [ "$source" = Other... ]; then
    source=`zenity --entry --title="Sync source" --text="Please enter the source path on local computer" --entry-text="$PWD"` || exit 1
  fi
fi

# normalize and remove trailing /
source=`readlink -f "$source"`
source=${source%/}

if [ ! -d "$source" ]; then
  zenity --error --text="$source is not a directory"; exit 2
fi

if [ $2 ]; then
  # TODO: support multiple sources
  zenity --warning --text="Only one directory can be synched, using $source"
fi

# find matching destinations from stored ones
destinations=""
for s in $sources; do
  if echo "$source" | fgrep $s; then
    dest=`fgrep "$s" $paths_file | awk -F'|' '{print $2}'`
    suffix=${source#$s}
    suffix=${suffix%/*}
    destinations="$destinations $dest$suffix" 
  fi
done

# ask user to chose one of the matching destinations of enter a new one
dest=`zenity --list --title="Sync destination" --text="Choose where to sync $source" --column=Destination $destinations New...` || exit 3
if [ $dest = New... ]; then
  basename=`basename "$source"`
  dest=`zenity --entry --title="Sync destination" --text="Please enter the destination (either local path or rsync's remote descriptor), omitting $basename" --entry-text="user@host:$(dirname $source)"` || exit 3
  echo "$source|$dest" >> $paths_file
fi

# check if user is not trying to do something wrong with rsync
if [ `basename "$source"` = `basename "$dest"` ]; then
  # sync contents of source to dest
  source="$source/"
fi

log_file=/tmp/Sync.log
rsync_opts=-rltEorzh
echo -e "The following changes will be performed by rsync (see man rsync for info on itemize-changes):\\n$source -> $dest\\n" > $log_file
( echo x; rsync -ni $rsync_opts --delete "$source" "$dest" 2>&1 >> $log_file; rsync_result=$? ) | zenity --progress --pulsate --auto-close --width=350 --title="Retrieving sync information" 

if [ $rsync_result -ne 0 ]; then
  zenity --error --title="Sync" --text="Rsync failed: `cat $log_file`"; exit 4
fi

num_files=`cat $log_file | wc -l`
num_files=$((num_files-3))

if [ $num_files -le 0 ]; then
  zenity --info --title="Sync" --text="All files are up to date on $dest"; exit
fi

zenity --text-info --title="Sync review ($num_files changes)" --filename=$log_file --width=500 --height=500 || exit 4

num_deleted=`fgrep delet $log_file | wc -l`
if [ $num_deleted -ge 100 ]; then
  zenity --question --title="Sync" --text="$num_deleted files are going to be deleted from $dest, do you still want to continue?" --ok-label="Continue" || exit 4
fi

rsync_progress_awk="{ 
 if (\$0 ~ /to-check/) {
  last_speed=\$(NF-3)
 }
 else {
  print \"#\" \$0 \" - \" files \"/\" $num_files \" - \" last_speed;
  files++;
  print files/$num_files*100 \"%\";
 }
 fflush();
}
END {
 print \"#Done, \" files \" changes, \" last_speed
}"

# note: delete-delay below means that any files will be deleted only as a last step
rsync $rsync_opts --delete-delay --progress "$source" "$dest" | awk "$rsync_progress_awk" | zenity --progress --width=350 --title="Synchronizing $source" || exit 4

Deleting thumbnails for inexisting photos

Freedesktop for some years already has a spec on how applications should manage image thumbnails (use Next link there). The spec is now followed by majority of Gnome and KDE applications, including F-Spot, which is one of the very few applications that uses large 256x256 thumbnails under ~/.thumbails/large.

The spec specifies to store thumbnails in PNG format, naming the files after the MD5 sum of the original URLs of the original files, eg 81347ce6c37f75513c5e517e5b1895b8.png.

The problem with the spec is that if you delete or move image files, thumbnails stay there and take space (for my 20000+ photos I have 1.4Gb of large thumbails).

Fortunately, you can from time to time clean them by using simple command-line tricks, as the original URLs are stored inside of thumbnail files as Thumb:URI attributes. I don't recommend erasing all of your thumbnails, because regeneration will take time.

In order to create a list of matching thumbnail-original URL pairs, you can run the following in a terminal inside of either .thumbnails/large or .thumbnails/normal directories (it will take some time):

for i in `ls *.png`; do
identify -verbose "$i" | \
fgrep Thumb::URI | sed "s@.*Thumb::URI:@$i@" >> uris.txt;
done

This will get you a uris.txt file, where each line looks like the following:

f78c63184b17981fddce24741c7ebd06.png file:///home/user/Photos/2009/IMG_5887.CR2

Note that the provided thumbnail filenames (first tokens) can also be generated the following way from the URLs (second tokens) using MD5 hashes:

echo -n file:///home/user/Photos/2009/IMG_5887.CR2 | md5sum

After you have your uris.txt file, it can be easily processed with any familiar command-line tools, like grep, sed, awk, etc.

For example, in order to delete all thumbnails matching 'Africa', use the following:

for i in `cat uris.txt | fgrep Africa`; do rm $i >/dev/null; done

So, as you can see, it is pretty simple to free a few hundred megabytes (depending on the number of thumbnails you are deleting).

With this kind of trick you can even rename the thumbnails of moved files if you use md5sum to generate the new filenames from the URLs, as shown above. This will save you regeneration time.

Announcing F-Spot Live Web Gallery extension

I am happy to announce a new extension for F-Spot, the popular Linux photo management application - LiveWebGallery. Once installed, invoke it from the Tools menu in F-Spot's main window.

The extension contains a minimal web server implementation that serves the user's gallery over HTTP and can be viewed with any web browser, even on Mac and Windows. So now you are able to easily share your photos with family, friends, colleagues no matter what operating system and software they use by doing just a few mouse clicks in F-Spot. The only requirement is that they have to be on the same network, or be able to access your machine's IP address in some other way.

As you can see in the screenshot, you can choose whether to share photos with a particular tag, current view in F-Spot (allows you to create an arbitrary query) or the currently selected photos.

To activate the gallery (start the embedded lightweight web server), just click activate button in the top-right corner. On activation, the URL of the web gallery will appear, allowing you either to open it yourself or copy the link and provide to other viewers.

After that all the options can still be changed in the dialog and will affect all new viewers or those pressing the browser's reload button.

Most of us already know that many pictures are rarely viewed after they are made (Point, Shoot, Kiss It Goodbye). F-Spot tries to fix this with its very powerful tagging features - tags make it much easier to find photos made long ago. This, however, is no magic - the possibilities of finding the right photos when needed depends on how well you tag. Now, this extension allows to make tagging even more useful, because other people can help you with the most difficult part - properly tagging can sometimes be a lot of work. With this extension, you can delegate some it to other people! The gallery is not read-only - if you choose so, an editable tag can be selected and viewers can add/remove this tag from photos (currently only in the full photo view). This is especially useful to let other people to tag themselves in your library. For security reasons, editing is disabled by default.

As time goes by, a lot more features can be added to Live Web Gallery extension, especially related to editing photo metadata (tagging, editing descriptions, flagging for deletion).

As far as I know, being able to share your photos on the local network without any software or OS requirements is a unique feature of F-Spot now. No other photo management application can do this to date.

Downloading

The source code is on Gitorious, live_web_gallery branch (until is has been merged to the mainline).

To install, use the Edit->Manage Extensions menu in the F-Spot, click on Install Add-ins and then Refresh. After that LiveWebGallery should be available under the Tools category.

Or, alternatively, you can download the precompiled binary and put it to:
~/.config/f-spot/addins or /usr/lib/f-spot/extensions

Note: F-Spot 0.6 is required for it to work. You can already find deb/rpm packages for F-Spot 0.6 or 0.6.1 for most distributions and it will be included in the upcoming distro releases this autumn.

Hopefully, the extension will later be distributed with newer versions of F-Spot by default.

Enjoy! Comments are welcome!

Use LightZone from F-Spot as external editor

In a previous post I have shown how to teach LightZone, a non-destructive photo editor, to open files passed from the command-line (for some strange reason it doesn't do it out-of-the-box).

Now, I want to be able to use LightZone as an external editor from F-Spot, which I use for my photo workflow.

F-Spot has a convenient Open With menu when right-clicking on a photo, we just need to add LightZone to this menu. After some research, it appear that F-Spot uses the standard desktop entry specification files to populate this menu. These files can be located either in /usr/share/applications or in user's home, ~/.local/share/applications. You can use either location, but I prefer the latter one, because I have unpacked LightZone into my home as well.

Here is the working lightzone.desktop file:

[Desktop Entry]
Version=1.0
Type=Application
Name=LightZone Photo Editor
Exec=LightZone %u
TryExec=LightZone
Icon=/home/USERNAME/LightZone/LightZone_32.png
Terminal=false
Categories=Graphics;2DGraphics;Photography;RasterGraphics;GTK;
StartupNotify=true
MimeType=image/tiff;image/jpeg;image/x-canon-cr2;image/x-canon-crw;image/x-nikon-nef;image/x-pentax-pef;

This file expects that you have followed the previous post and already have LightZone in the PATH which accepts a filename to open on the command-line.

• Exec - this line specifies what command to run, %u means to pass the selected file's URL on the command-line. For some reason with trial and error, I found that F-Spot is only able to pass URLs like this, specifying %f doesn't work with F-Spot. But if you look at the LightZoneOpener.java source, you will see that it supports URLs as well as filenames.
• Icon - change this one to the full path of LightZone's icon, it supplied in the original archive.
• Categories - this specifies where LightZone will appear in the ''Applications'' menu.
• MimeType - here you must list all image mime types that you want LightZone to open. This is especially important for RAW files. As I own a Canon camera, I have most of my photos in CR2 format, so I need to be sure that image/x-canon-cr2 is in the list. I have also specified CRW, NEF and PEF mime types for older Canon, Nikon and Pentax cameras, respectively. These mime types are already registered in Ubuntu Intrepid (not sure about other distributions). Here is some info on how to register new mime types in Gnome, in case your camera's format is not registered yet.
  

After chosing the Open With->LightZone, F-Spot will ask whether to create a new version for the file. Select 'No' - this won't work with RAW files and LightZone anyway, because LightZone saves files automatically with the _lzn.jpg suffix and F-Spot doesn't know about it.

Getting this right requires patching F-Spot (I am going to do that later). For now, you will have to import the saved file manually, if you want it to appear in F-Spot.

Opening files with LightZone from command-line

LightZone is a very useful commercial photo editor with some unique features like non-destructive and layer-based editing. To my mind, the developers has taken a very clever approach to save resulting edits inside of smaller-size JPEG files (thumbnails), so that any program can be used for previewing the resulting image, but opening of the file in LightZone will load the original image and show all the edits again with the possibility to make any changes and export a full-resolution image. The cool thing here is that edited files are very small especially when compared to 10Mb+ source RAW photos, contain all the editing history and can be previewed quickly with any software. And all this runs on Linux (thanks to Java - write once, run almost anywhere).

The only problem with LightZone (at least the Linux version) is that it doesn't accept filenames from the command-line! You have to start the program and select the file manually using the embedded file browser. Of course, this is not an option if you want to run LightZone from another application as an external editor, eg from F-Spot (more on this in a later post).

To make a long story short, I have written a small Java program that takes a filename on the command-line and then modifies LightZone preferences, so when you run it next time it will directly open the specified image.

Here it is:

import java.io.*;
import java.net.*;
import java.util.prefs.*;

/**
 * LightZoneOpener - will modify LightZone preferences to open the 
 * specified file (image) on next startup.
 * This is useful to force LightZone to open a particular file from 
 * the command-line, just run this code before starting LightZone. 
 *
 * @author Anton Keks
 */
public class LightZoneOpener {

  public static void main(String[] args) throws Exception {
    if (args.length != 1) {
      System.err.println("Please specify filename to open in LightZone");
      System.exit(1);
    }
    String filename = args[0];
    if (filename.startsWith("file:"))
      filename = new URI(filename).getPath();
    File file = new File(filename).getCanonicalFile();
    if (!file.exists()) {
      System.err.println(file + " doesn't exist!");
      System.exit(2);
    }
    File fileDir = file.getParentFile();
    
    // set image folder as current one
    Preferences folderPrefs = Preferences.userRoot().node("com/lightcrafts/ui/browser/folders");
    int i = 0; File dir = file;
    while ((dir = dir.getParentFile()) != null) {
      folderPrefs.put("BrowserTreePath" + i++, dir.getName().isEmpty() ? "/" : dir.getName());
    }
    folderPrefs.remove("BrowserTreePath" + i);
    
    // set selected image in the current folder
    Preferences appPrefs = Preferences.userRoot().node("com/lightcrafts/app");
    appPrefs.put("BrowserSelectionMemory" + fileDir.getPath().hashCode(), file.getPath());
    // tell LightZone that last startup was OK just in case
    appPrefs.put("StartupSuccessful", "true");
    
    System.out.println("LightZone is now ready to open " + file + " on next start");
  }
}

Here is what to do:

1. save it to LightZoneOpener.java
   
2. compile with javac LightZoneOpener.java
3. run with java LightZoneOpener

Then you can create a small script that will automate the stuff for you (save it to ~/bin/LightZone):

#!/bin/bash
java -cp ~/bin LightZoneOpener "$@"
~/LightZone/LightZone

This assumes that you have extracted LightZone to ~/LightZone (in your home dir) and have the following two files in the ~/bin dir: the compiled LightZoneOpener.class and the script file LightZone (don't forget to set the execute permission with chmod a+x ~/bin/LightZone)

Here is all this pre-compiled. Just extract the file directly in your home and it will put all needed files into the bin directory. After next login your local bin will be in $PATH, so you will be able to use it.

Now you can run LightZone filename on the command-line in Linux (or using Alt+F2)! Have fun!