Setting up backup of photos to Flickr using iphoto-flickr

Update: Photos and better options

Apple's new Photos app replaces iPhoto, and the script I describe on this page does not work with Photos. However, I discovered the new Flickr uploadr - a complete replacement wrapped in nice graphical pointy-clicky goodness and vastly simpler to set up. Go have a look, and expect a lengthy initial scan before actual uploading starts.

Update: Did it break for you?

Did you come here because you followed these instructions before July 1st 2014 and things have now stopped working? You need to update the script to the latest version so that it uses SSL. Download it again replacing your old version. Or, if you want to hack, add FlickRaw.secure = true as line 73. The devil is in the details …

Back to the article

A while ago I discovered iphoto-flickr, which is a small script to back the entire contents of iPhoto up to Flickr. Flickr is a long-lived photo service and web site which recently started offering one terabyte of storage for free, making comlete photo backups a rather tempting idea. I found the script, got it running on my computer and went on to tip people off about it when I appeared on the Appsnack podcast. A listener then asked if I could elaborate a bit on actually getting the thing running, so here goes. Some familiarity with the command line (Terminal app) is expected.

A warning about automation - there be dragons

If you plan to run iphoto-flickr automatically using some kind of scheduled task, you may want to read my notes on that below before starting to follow the instructions. In short, I do not think I have the neatest (or even a whole) general solution for automatic running of the script. I will write about what I did in this area, but it is not a complete story or guide.

Getting the script and running backups

• Have a Mac
• Have a Flickr account
• Request an API key and secret from Flickr. A "non-commercial" key works great. You will need to feed these into the script the first time you run it, so keep them nearby. …
• Change your default settings on Flickr so that photos are private by default … unless you really want everyone to see your whole photo library of course …
• Have the right version of Ruby (1.9 or higher). On 10.9 (Mavericks) this is apparently already set up. Type
  ruby -v
  in Terminal and see what it says.
• Terminal time!
  • Open the Terminal app or whichever equivalent you prefer.
  • Create a folder where you want the script to reside.
  • You can put the script wherever you like, but if you want to automate things or otherwise invoke it regularly it is good to choose a location which you will not change by accident. So, maybe not your desktop, downloads folder or other place which you might go through and "clean up" one day.
  • Then follow the instructions from the project readme:
    • sudo gem install flickraw-cached
      installs stuff used by the script to run.

    • git clone https://github.com/jawj/flickrbackup.git

      This downloads the script itself to the folder you are in.

    • chmod u+x flickrbackup.rb

      This makes the script runnable, i.e. actually useful for our purposes.
• Do some initial runs
  • ./flickrbackup.rb
    Hit enter and watch it go. Fill in the information it asks for.
  • The first few (okay, many) times it failed for me. Seems as if some operations just took too long and timed out. Fortunately the script is able to pick up where it left off, so eventually it got through and started uploading.
  • If you want to speed things up, you may want to ensure the computer doesn't go to sleep while the initial upload is running.
  • Future uploads - simply running the script again - will only send any new images.
• At this point you can run the script whenever you like to back up the latest photos.

Victory!

Of course, it's not a good backup if you need to rememeber to run it yourself. The trick is to make your computer do it for you. There are many ways to do this, and any solution which works for you is by definition good. What follows is my story about what I did to solve this. I half assume that I have missed important points below, and I am sure that there are cleaner ways of accomplishing this. Consider the following more of a story, inspiration or a collection of thoughts than a step by step guide.

The automation story

I decided to use launchd to set up the script to be run regularly. That is kind of the geeky low-level way to go. Any scripting or timing solution which works for you and happens without intervention is by definition good. I messed around a great deal to get this working, probably did things in non-recommended ways and did not take notes as I was doing it. I would love to hear from people who know of better ways to get things done.

• I have a copy of flickrbackup.rb in /usr/bin. It is there to make it as easy as possible to launch from launchd. I should probably have used a link to another location for the script, or added the script to my path in some other way, but this is the way it ended up as I was plowing through and trying to keep things straight to myself as I was going. This copy is owned by root and the wheel group, a change which I am not sure why I made either. Anyway, in this version of the script, I have edited dataDirName on line 14 to point to my users' flickrbackup folder in /Users/fredrik/Library/Application Support. Note that this path can not use ~ to refer to the home directory as launchd will be running the script as a different user. All this is so that the script will keep running with the same settings and caches it built up during initial runs. If I did not do that, it would start uploading everything again as it had no idea an old copy had already done so. If you only run the script through launchd I would imagine you don't have to do this.
• Create a launchd plist for the backup.
  • This file simply describes a task to be run regularly - flickrbackup in this case - and how often it will run. I searched, cut, paste and generally messed around a lot as I created my file and thus feel completely unqualified in providing any guidance for the creation process. The contents are pasted at the end of the page though.
• Place the plist in /Library/LaunchDaemons
  • Things placed here will run as root, and no user has to be logged in for them to run. I wanted this. Otherwise, each user has a corresponding folder in their own library where plists can be placed to run as that user provided he or she is logged in.
• Changed the owner of the plist to root:

  sudo chown root path/to/fileName.plist

  I think this mattered, but I would not bet anything on it.
• Load the plist into launchd:

  sudo launchctl load path/to/fileName.plist

• Sit back, wait for the set time to occur and marvel as your photos show up on Flickr.
• Or, tell launchd to run the script at once to check that it worked.
• When the script runs, any output can be viewed in the Console application.

My launchd plist file

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.bjoreman.flickrbackup.backup</string>
    <key>ProgramArguments</key>
    <array>
        <string>flickrbackup.rb</string>
    </array>
    <key>StartCalendarInterval</key>
    <dict>
        <key>Hour</key>
        <integer>10</integer>
        <key>Minute</key>
        <integer>0</integer>
    </dict>
    <key>StandardErrorPath</key>
    <string>/tmp/flickrbackup.err</string>
    <key>StandardOutPath</key>
    <string>/tmp/flickrbackup.out</string>
</dict>
</plist>