schussman.com logo

Sync Lightroom Galleries to Android - Automatically!

Now that I have a photogenic, tiny gurgling creature in the house, I’m shooting a lot of photos, both on the Droid X and the Pentax. Most of these photos end up, one way or another, in Lightroom for cataloguing, and as I’ve described earlier, I have a nice workflow for updating these rapidly-growing galleries on my iPad.

But how about the Droid X? It has a nice screen, and I want to foist photos of my beautiful boy on anyone I happen across — so I started to wonder if, continuing to use Lightroom as my core platform, I could keep a small gallery of photos on the Android phone with as little manual work as possible.

Here’s the executive summary: I again use a smart published collection in Lightroom to create the gallery; then I use a LaunchAgent in OS X to monitor the mount point of the Droid X on the filesystem; and a tiny shell script syncs the published gallery to the Droid whenever I plug it into the MacBook Pro. Read on for altogether too many details.

First, a couple of things to note about how the Droid X generates image galleries [ note that this may apply to all Android devices; I have no idea, and your mileage may vary ]:

  • Android automatically displays galleries based on image type — so you don’t need to update a database or anything to build your galleries on the phone. You just need folder(s) full of images.
  • When you run the Gallery app, it will display galleries with the name of the parent folder containing the images. This means you can make a folder tree on the Droid’s SD card and neatly package multiple gallery folders within it, without cluttering up your root directory.

Prepare the SD card

When I attach my Droid X via USB to the MacBook Pro, it automatically mounts it as a volume titled NO NAME, so I used the Finder to change the label to DROIDX. This, happily, seems not to have affected any of the phone’s operations; the Droid must not depend on the name of the SD card for any of its internal work.

The second preparation step is to create a master gallery directory on the SD card. This is just for housekeeping purposes; everything in the galleries will be in a subdirectory of that top-level directory. Again, I simply did this with the finder: navigate to the DROIDX drive when the phone is connected, and create a new top-level directory, Galleries.

Your SD card and Droid should now be ready to go.

Set up the Gallery/Galleries

Just like last time, I’m using smart publish collections, based on keywords, to populate the galleries that will be synced. I decided to prefix all the keywords for this usage with dx — so, dx-gallery is my main tag, and I’ve edited it in LR to not be included in export. It’s a housekeeping tag only.

I’ve tagged a bunch of images with the dx-gallery tag.

Then it’s off to build the publish service. Here’s what it looks like in the publishing manager:

And add to that service a Published Smart Set that looks like this:

You can assign multiple keywords to the Smart Set, or create as many smart sets as you want galleries, and use unique keywords to assign photos to each. Each smart set within the published collection will appear in your directory tree as a subdirectory of the top-level folder for the set — and that makes the syncing to the Droid easy.

Automate syncing via LaunchAgent

Everything I know about LaunchAgent I learned from this tutorial. This is basically a concise repeating of that description, edited for our purposes. (I’ve previously described using this process to perform system backups)

First, if it’s not already there, mkdir ~/Library/LaunchAgents. This is the folder that OS X will watch for scripts triggered by system events such as mounting an external drive, which is exactly what plugging in the Droid does.

In that directory, make a new plist file (I called mine droid-sync-watch.plist) and give paste this into it:

<?xml version=1.0 encoding=UTF-8?> <!DOCTYPE plist PUBLIC “-//Apple Computer//DTD PLIST 1.0//EN” \ “http://www.apple.com/DTDs/PropertyList-1.0.dtd”> <dict> <key>Label</key> <string>droid-sync</string> <key>LowPriorityIO</key> <true/> <key>Program</key> <string>/Users/alan/Library/Scripts/droid-sync</string> <key>ProgramArguments</key> <array> <string>droid-sync</string> </array> <key>WatchPaths</key> <array> <string>/Volumes</string> </array> </dict> </plist>

In short, this plist file tells the OS to run the identified script (~Library/Scripts/droid-sync) when the specified WatchPath changes.

The sync script itself lives in ~Library/Scrips and consists of a check against the desired volume (here’s where naming the Droid SD card comes in) and an rsync of the designated published collection to the previously-generated target directory on the SD card.

#!/bin/bash # delay a short time to make sure the path is available echo -n "[*]-- new /Volumes... sleeping" | logger sleep 20 if [ ! -e "/Volumes/DROIDX" ]; then       echo -n "[*]-- DROIDX NOT connected - Exiting" | logger       exit 0    else       echo -n "[*]-- DROIDX Connected - Performing gallery sync" | logger fi # rsync with delete option rsync --delete -r ~/Pictures/Exported\ Photos/Droid/ /Volumes/DROIDX/Galleries echo -n "[*]-- DROIDX gallery sync complete" | logger

Finally, tell the LaunchAgent controller to watch your scripts by doing the following at a terminal:

  • launchctl load ~/Library/LaunchAgents
  • launchctl list | grep sync

You should see your droid-sync script appear in the list produced by the second command, above.

Now you’re ready!

Plug in and go

That really should do it. When you connect the Droid X, you can watch the messages in the sync script appear in your console log, and everything should work — after syncing, disconnect the device again and fire up the gallery application; in the “Folders” section of the gallery you should see an item for each gallery you create in Lightroom. You’re done!

As with the iPad workflow, the great part of this is that once it’s set up, it’s pretty automatic. If you edit, add, or remove photos, as long as you republish the collection, those changes will all be pushed over to the Droid. Thanks to a little nerdery and Lightroom, we’ve built a bit of functionality that the Droid software doesn’t directly offer — and it’s still done with LR as the core of the photo library.

Misc & Etc

What’s in the bin this week?

  • your.flowingdata — a personal data aggregator in the same vein as daytum and mycrocosm. Flowing Data is a great site, so this tool has lots of promise. Update activities via Twitter dms.
  • Summertime in Flagstaff is just great. (The Schultz fire is now fully contained and crews are working on flood/mudslide prevention.)

Downtown after dinner

  • I’m still loving the iPad. My Lightroom 3 to iPad gallery workflow has proved to work pretty well for my needs. I’m eagerly awaiting the release of PlainText and have started to daydream about an app or web utility that would facilitate writing markdown, textile, and other markup-formatted text.
  • Shouldn’t people writing fake term papers for sale just expect that they’re going to get ripped off by douchebags?

Lightroom 3 to iPad gallery workflow

(Or, an efficient way to use Lightroom to manage photos on your iPad)

I got myself an iPad a few weeks ago. Short review: It’s a big iPhone only in the sense that an iPhone is just a phone; that is, it’s hawt and I love it. (More ZOMG iPad articles to follow, I’m sure.)

The iPad really shows off photos, so I’ve been thinking about the best way to manage the photos that I want to put on it. With the non-beta release of Lightroom 3 this week, I think I’ve found a start at a workflow in the new publishing service that it offers.

First a note about syncing photos to the iPad. Like the iPod/iPhone, you have to select a single parent directory, and optionally subdirectories, of photos that you want to sync. The iPad stores photos in a series of albums simply named after the subdirectory where they live on your hard drive. In my case, I keep the photos that I publish out of Lightroom in one of a number of subfolders of a ~/Pictures/Exported Photos directory, but I don’t necessary want to sync all of them to the iPad. And, for the iPad, I wanted a little more control over the album naming for purposes of navigation — so I didn’t want to keep the directory names I’ve previously used for simple file management.

So: I have an “iPad” directory in my exports folder, and that’s what I sync to the iPad — along with all its subdirectories. I’ve copied a handful of existing albums into that directory, which isn’t a great solution but it does work for now. Apple has apparently decided that symbolic links cannot be synced [that would be a great capability, wouldn’t it?], so I can’t simply create links to the file locations of the existing, non-Lightroom albums that I want to sync.

iPad sync directory; Lightroom 3 publish service directories are subfolders of this

But the Lightroom albums are another story. Right now, I’ve simply created a couple of Hard Drive publish services, each of which publishes to a subdirectory of that “iPad” folder I described above. For example, the current flickr one simply sets up a set of export parameters for photos that will be published to a subdir.

Lightroom 3 Hard Drive publish service setup

But the slick part comes by making a smart collection that will publish everything I post to flickr in 2010. Simply build the smart collection within the publish service:

Creating a published smart folder in Lightroom 3

And then set it to include all photos that match some key criteria: Since I use Jeffery Friedl’s flickr plugin, I can call directly on the metadata it creates at export:

Published Smart folder settings to include a subset of Flickr-published photos

I now have a smart collection within a publish service, so the photos that I export that match that criteria will automatically be added. I’ll have to occasionally republish to the service itself; and each time I do so, the iPad will gather up those photos the next time I sync.

You can use the same methodology to set up hard drive publish services for any album you like; each service will appear on the iPad as an album, and if you create a smart collection (or a series of them) as part of the service, then they’ll be updated automatically as you perform you regular photo workflow.

One final note on why I think the publish service is an ideal tool for this workflow, rather than a solution such as simply exporting to a folder: The publish service will apply future changes to the service settings to all photos under its control. So, were I to decide to apply a watermark, a border (via, for example, LR/Mogrify 2), or adjust photo quality in order to save space on the iPad, those changes made to the publish service would result in the option to re-process the current photos:

Publish service allows for the updating of all published photos when changes are made to the service

In a sense, then, the photos produced via the publish service are truly synced — future changes will be managed and will trigger re-processing, if I choose. I think that’s pretty Good Stuff.

Another year of photo data!

Following in the modest two-year tradition I’ve established (see 2007 and 2008 posts), here is my 2009 photo data from my Lightroom catalog!

[ quick howto: Lightroom 1 & 2 (and 3) databases are in sqlite3 format, which means that freely-available tools can extract data from them. I use sqlite3, some shell scripting, and R (and occasionally excel) to produce summaries of that data. Why? Data offers some insight into the kinds of photos I take. Mostly, though, it’s fun. I’d be happy to expand on the actual code that goes into these plots, if there’s interest. ]

Below is a set of plots that summarize some of this year’s data. Click through to flickr to see the larger version.

2009 photo data!

What’s interesting this year? Well, crop ratios looked pretty similar to last year, so this year, for the first time (suggested in a post by Keitha, whose photos I admire tremendously, and whose Pentax lens set I envy with the fire of a million anti-glare-coated nine-aperture-bladed all-metal suns) I pulled out some information about aperture for each of the prime lenses that I shoot with. You can see these four frequency plots (for each of the Pentax DA 70mm F2.4 ltd, FA 50mm F1.4, FA 35mm F2.0 and DA 21mm F3.2 ltd lenses) in the left hand column of the image. Right off the bat you can see that I shot a lot with the FA 35mm this year (which is confirmed by the “overall lens use” plot on the right column). In fact, I took that lens along as my sole lens on a few long weekend trips to Ventura, CA, and the San Juan Islands, and really loved its performance. It does great at large apertures, but I also used it a lot for street shooting at f/8 and smaller apertures.

Runner-up in frequency this year is the FA 50mm F/1.4, which ordinarily I would say is my favorite lens (and it very much still is; it just wasn’t as convenient a focal length to take as my only lens on those vacations). Its sweet spot [where it’s sharpest but still has a nice narrow depth of field] is about F/4, which is where I primarily use it.

Neither the DA 70mm F/2.4 or the DA 21mm F/3.2 got as much use this year, but I really love some of the photos I took with those lenses. In fact, I carried these two lenses specifically for their light weight and trim size on the Flagstaff photowalk I organized in July.

Car / Cat Ranch house / wide Crow Pomegranite Backside Doorman

How did 2009 stack up to 2008? In terms of absolute frequency, nearly identical! I kept 1308 frames last year, compared to keeping 1340 in 2008. Far fewer of those are picks, or posted to flickr — though a good number are waiting for me to come back to, to finish workup or to make a print.

And that’s it for the 2009 photo stats! I did re-work my keyword network code, so perhaps can follow up this post with a little more about keyword relationships.

If you’d like to know more about extracting and summarizing info from your own Lightroom catalog, please let me know (and check out my other lightroom-related posts)

And, as last year, I hope soon to follow up with a report on my 2009 photo goals, and to set a few for 2010.

Lightroom: Build thumbnails and full-size exports simultaneously

It would be great if Lightroom could string export presets together, to perform operations sequentially or simply to perform several processes on one image to produce several different outputs.

The first use I’d put to this functionality would be generating thumbnails of exported images right along-side the full-sized image — a common task when producing images to upload to a blog, for example.

Absent that kind of flexible functionality, here’s a nice solution for generating thumbnails of images and full-size exports at the same time. Use Jeffrey Friedl’s Run Any Command plugin in conjunction with mogrify: Just insert “run any command” to an existing export preset, and then add the following as the Command to Run (assumes you have LR2/Mogrify installed, which of course you do):

cp '{FILE}' '{NAME}'-small.JPG ~/Library/Application\ Support/Adobe/Lightroom/Plugins/LR2Mogrify.lrplugin/LRMogrify.extras/mogrify -resize 50% '{NAME}'-thumbnail.JPG

The command makes a copy of your original image (which is available to “run any command” plugin via the {FILE} token), with “-small” appended to the filename, and then uses mogrify to resize the copy to 50%. You can optionally adjust the percent to scale or add a “-quality” option to the mogrify command.

You could perform this command more cleanly, without the copy step, if you have ImageMagick installed independently of LR2/Mogrify: The full IM package includes the “convert” command, which allows for output to a new filename. Since I didn’t want to separately install ImageMagick, I just make a copy of the file and then invoke mogrify via the path to the plugin.

Presto! Instant thumbnails, appropriately named, to accompany your export. Pretty good stuff, made possible by the great work of Jeffrey Friedl. Check out all his lightroom goodies.

Lightroom tips keep coming

Between Scott Kelby and John Beardsworth, these two lists of Lightroom tips provide lots of useful pointers:

Kelby’s tip about “solo mode” is pure gold. I’ve been using it for a while now and it’s a super way to work. I’m not sure about the keywords tips, though: Both Scott and John caution against keywords and nesting keywords; while I agree that finding a way to use consistently use keywords (and metadata in general) is important, I haven’t run into problems of too many of them, quite yet. Periodically taking a keyword inventory and identifying duplicates has helped keep my catalog relatively clean (and me relatively sane).

Today's Lightroom quick tip: Targeted adjustment shortcuts

It’s nice to keep learning things about Lightroom. It tends to have solutions for problems long before I realize that I need them, and so I was happy to find this morning that it has handy shortcuts for all the targeted adjustment brushes.

  • shift-alt-cmd-N: None (cancel adjustment)
  • shift-alt-cmd-T: Tone adjustment
  • shift-alt-cmd-H: Hue adjustment
  • shift-alt-cmd-S: Saturation adjustment
  • shift-alt-cmd-L: Luminance adjustment
  • shift-alt-cmd-G: Grayscale mix

Activating any shortcut will jump your develop panel to the appropriate controller, so you can keep your cursor focused on the image and still see the effects of your adjustments in the sliders or levels. It’s a nice, efficient way to minimize extra mouse movement and make quick modifications.

 

Check out the rest of my Lightroom posts for much, much more, including more database tinkering, keywording, and workflow.

What's in your Lightroom 2 catalog?

A question recently came up over in the Flickr Lightroom group about generating a list of all filenames or selected filenames in a catalog. There’s a pretty nice straightforward bit of SQL that will produce just such a list.

My new favorite tool for poking around in the Lightroom database is the SQLite Manager extension for Firefox. It plugs seamlessly into Firefox and is a very good tool—faster than the old sqlite3 browser, and with the added bonus of not requiring Rosetta to be installed under Snow Leopard. Plug the following code into the “Execute SQL” box (or kick it old school via the command line sqlite3 tool):

SELECT pathFromRoot, baseName, extension FROM AgLibraryFile JOIN AgLibraryFolder ON AgLibraryFile.folder = AgLibraryFolder.id_local

You can do even better by using the sqlite3 concatenate operate in the select statement:

SELECT pathFromRoot || baseName || '.' || extension FROM AgLibraryFile JOIN AgLibraryFolder ON AgLibraryFile.folder = AgLibraryFolder.id_local

That will produce a nicely-formatted list of files:

	2007-02-19/IMGP1609.PEF
	2007-02-19/IMGP1610.PEF
	2007-02-19/IMGP1611.PEF
	2007-02-19/IMGP1612.PEF
	2007-02-19/IMGP1613.PEF
	2007-02-19/IMGP1614.PEF

Still building, we can add in the absolutePath field to get a complete path — very useful for when you have files stored in multiple locations, such as offline storage or backup locations.

SELECT absolutePath || pathFromRoot || baseName || '.' || extension FROM AgLibraryFile JOIN AgLibraryFolder ON AgLibraryFile.folder = AgLibraryFolder.id_local JOIN AgLibraryRootFolder ON AgLibraryFolder.rootFolder = AgLibraryRootFolder.id_local

And, finally, because how often do you really need to see the paths for all of the files in your library, really?, we can limit the query to just files in a quick collection. This takes a couple of extra steps, because we can’t directly get to the keyword data from the path data:

SELECT absolutePath || pathFromRoot || baseName || '.' || extension FROM AgLibraryFile JOIN AgLibraryFolder ON AgLibraryFile.folder = AgLibraryFolder.id_local JOIN AgLibraryRootFolder ON AgLibraryFolder.rootFolder = AgLibraryRootFolder.id_local JOIN Adobe_images ON AgLibraryFile.id_local = Adobe_images.rootFile JOIN AgLibraryTagImage ON AgLibraryTagImage.image = Adobe_images.id_local WHERE tagKind like "AgQuickCollectionTagKind"

And there you have it. One more fun trick you can pull thanks to the sqlite3 database that underlies your Lightroom 2 catalog.

 

Check out the rest of my Lightroom posts for much, much more, including more database tinkering, keywording, and workflow.

Always more to learn about Lightroom

I’ve been reading the comments thread of this great post at Scott Kelby’s Photoshop Insider blog. It’s absolutely chock full of concise tips for getting more out of Lightroom, and it drives home to me that there’s always lots, lots more to learn.

Here’s my favorite new-to-me tip (Thanks, Scott!): Did you know Lightroom 2 has a ‘solo mode’? It’s a really fantastic way to eliminate scrolling up and down through the set of side panels. Ctrl-click beside a panel’s name and select ‘solo mode’ to turn it on, and then only the currently-selected panel will be expanded. Select any other panel’s name to auto-expand it while collapsing the prior panel. In conjunction with using cmd-1 through cmd-7 to toggle panels, it’s (to me) a nearly unbeatable way to navigate the panels with minimal mousing. (Note this works in both Library and Develop modes.)

Lightroom antique presets

A couple of years ago I came across Camiel Schoonen’s very well-done set of antique presets for Lightroom/Lightroom 2 (link to flickr discussion page for the presets). They achieve some nice effects, as in my example below from a ride on the Grand Canyon Railway. The modified photo is on top, original beneath:

(link is to flickr photo)

After a laptop transition gone awry, Camiel lost his copies of the presets, but I still have a copy. I’ve forwarded these back to Camiel so he can resume their distribution over at his own site.

Thanks for the great work, Camiel!

Relative sized borders in Lightroom 2

I’m sure the rest of you figured this out ages ago, but it was a new realization to me. In addition to supporting multiple, different borders per image, the LR2/Mogrify border options for Lightroom 2 allow for borders to be applied on three metrics: pixel size, percent of height, and percent of width. That allow, for example, for easy application of letterbox-style top and bottom borders without needing to pay any attention to the cropped dimension of your image.

Screenshot - selecting fixed-percentage borders in Lightroom 2/LR2 Mogrity

The plugin default is in pixels; just switch to percent and you’re off. Something in the range of 10-12% of height seems to suit my eye pretty well.

Lightroom archiving and backup revisited

A while back I described my Lightroom 2 backup routine. A couple of recent blog posts around the web have given me some new things to think about with regard to backup and workflow.

Thomas Hawk writes about his workflow and Eric Scouten follows up with some thoughts. A core part of Thomas’s workflow is quick development of any and all keeper photos from a shoot and immediate export of those images to JPG. He re-synchronizes those JPGs with his Lightroom library and moves all the raw images offline (to a drobo; Jon at blurbomat extolls the drobo’s virtues, too) , and doesn’t apply keywords and such except to the finished JPGs.

Reacting to this, Eric notes a couple of things that get my attention (his emphasis):

  • The act of choosing what’s in and what’s out becomes an affirmative process rather than a process that’s about rejecting photos. Why not make the selection process a happy one?
  • This means the “selects” catalog is always in tip-top shape whenever I need to show someone my current work. Right now, the “main” catalog always contains some number of rough photos that haven’t been filtered out to the archive catalog.

By quickly moving from raw to selected and and “finished” JPG, the process builds the catalog of preferred images and gets the unused images out of the way. And applying keywords and other metadata only to the keepers dramatically reduces the overall metadata workload.

Where does this point me? I have a couple of thoughts:

  • Moving raw images offline, leaving only the JPGs readily-accessible, is a great way to impose some discipline over the “ooh, I might use that one someday” tendency. On the other hand, I’m not shooting in volume like Thomas is; while storage is an issue, there’s definitely a creative trade-off here. Until I shift my raw images to my network drive (usually about monthly), I’ll revisit a folder half a dozen times or more to see if there’s something interesting there that I didn’t see before. Thomas’s workflow, and that considered by Eric, are very different from mine though there is something appealing in it.
  • What this really opens up for me is a little bit of thinking about the intent and purpose of the photos that I take. Making sense of workflow and eventual archiving is a way to be conscientious of why I take photos. Is it because I’m an archivist? Is it for artistic purposes? Is it because I want to build a portfolio? I suppose I have a streak of all of those in me. I like the shooting itself, and I like the workup in Lightroom as well. So for me, it doesn’t quite make sense to workup photos quickly and move them offline.
  • However, it does make sense to develop a solid “keepers” collection as a sort of middle road; this collection gets updated frequently, but whether to keep those images as raw or JPG isn’t something I’ve quite sorted out.
  • What about those “someday” images? I need to consistently flag images with some potential and return to them regularly to re-evaluate.

Thoughts? How do you manage the possibility of future image development with your goals, limited creative time and storage space?

Eons

From today’s hike on Sedona’s Wilson Mountain trail:

Eons

Pentax K100D / Pentax SMC FA 35mm @ f/11. Black and white conversion with Lightroom 2.

Quick and easy curved borders in Lightroom 2

The post-crop vignette function in Lightroom 2 has much more flexibility than the comparable tool in Lightroom 1 — it’s made to allow for creative use of vignette rather than simply correcting lens vignetting. And you can put it to use to create quick and easy curved frames without calling on any external tools or plugins like LR2/Mogrify.

Sorels

The above image has a white curved border applied by using the following vignette settings:

Lightroom 2 -- post-crop vignette settings

The trick to the curve is the right combination of midpoint and roundness — adjust to taste. And the hard edge of the vignette comes from setting “feather” to zero. To make a black border? Just slide “amount” all the way to -100.

You can make a preset for both white and black versions: Click the big plus button in the preset pane, and select only the post-crop vignette setting to save. Do one for both black and white borders, and you’re good to go.

Quick and easy!

Photo Goals 2009

I really like this short set of new year Lightroom resolutions over at the O’Reilly Lightroom Blog. I have lots of room to improve my own use of Lightroom specifically, and my approach to my photography in general. I do think I have a good start at some of it, and I’ve really enjoyed writing about my experiences with Lightroom: Specifically apropos of Gene’s article, I’ve written a bit about my approaches to Lightroom file organization, cleaning up duplicate keywords, and adopting a backup routine. It has also been very rewarding to get nice feedback from other LR users on those articles. I’m looking forward to continuing my Lightroom writing projects in 2009.

Beyond the software/organizational side of things, I’ve been thinking the past few days about other elements of my photography that I want to develop [aargh] in the coming months. I’m hesitating to call these “resolutions,” but perhaps that’s what they are. In no particular order, here are a few of them.

  • I’d like to take more photos of people. There’s a vibrance to portraits that I love to encounter in other peoples’ photostreams, and I don’t get much practice at it. H tolerates my saying “just a couple more” for only so long, after all.
  • Related to the above, I would love to make my photography a more social endeavor. Flagsbürgtown should offer lots of possibilities for this, but I haven’t really taken advantage of them. My first attempt at generating some interest in a photowalk via the local flickr group garnered exactly zero responses, but I’ll try, try again.
  • Seems like as good a year as any to start learning lighting and flash, doesn’t it?
  • Creative vision, etc, etc.

I kind of yadda-d that last one, but it’s probably the most important one to me. I’d really like to start turn my photography into something more personally expressive. To tell the truth, I’m not sure what that means, but I do think it’s time. This may mean I shoot more but post fewer photos, or identify some projects to take on — who knows? I’ll keep you posted.

To closeout, here’s a set of a few of my own favorite photos from 2008. No particular criteria, just a couple (or so) per month that I particularly liked. Click through to the flickr page for the full sized version and links to the individual photos.

A few faves from 2008

Happy new year!

The year in Lightroom, by the numbers

I started last year to play with pulling data right out of my Lightroom catalog. How fun to combine interests in photography with my need to make data out of things. Last year about this time I posted some 2007 photo stats, and with the release of Lightroom 2 I came up with some keyword network maps of my flickr images.

Over at The Online Photographer, Marc Rochkind did some writing about meta metadata and released a tool for OS X that produces much more summary information than I had previously considered: His tool produces by-lens statistics on cropping and aspect ratio in addition to focal length usage. This generated some thoughtful conversation about composing in the viewfinder versus cropping, and Marc’s work spurred me to think more about my own stats, and so I went back to my own Lightroom 2 catalog with the sqlite browser and R to see if I could reproduce for myself some of the more interesting data that Marc’s tool generated. After some tinkering, I think I have a functional, reusable set of R tools for generalized reporting of Lightroom image data.

Like Marc’s ImageReporter, I can filter by image type, picks, ratings, Quick Collection, camera model (though this matters less for me since I have one P&S and one DSLR) and time period, and I added filtering by color label as well — hey, just for fun, even though I don’t use the color labels (I generally get rating fatigue using anything more than picks.)

So, what do I have? First, a reproduction of the stats I checked out last year: Monthly photos and focal length:

The year in Lightroom

I continue to primarily use my prime lenses, and my picture-taking appears to have notched down dramatically as compared to 2007. This is partly because of work, of course, but also because I’ve become much more selective about what I actually keep in the catalog.

We can break out focal length a bit more. For the two zooms that I use on my K100D, what are the mean focal lengths?

> lensFL [1] 5.8-23.2 mm 15 [3] 85.0 mm f/1.8 85 [5] smc PENTAX-DA 18-55mm F3.5-5.6 AL 31 [7] smc PENTAX-DA 21mm F3.2 AL Limited 21 [9] smc PENTAX-DA 50-200mm F4-5.6 ED 121 [11] smc PENTAX-DA 70mm F2.4 Limited 70 [13] smc PENTAX-FA 35mm F2 AL 35 [15] smc PENTAX-FA 50mm F1.4 50

So that’s kind of interesting, suggesting that I use the 200mm zoom at about the middle of its range. But the mean isn’t necessarily informative. Here’s a plot of focal length for one of those zooms:

Focal lengths plot, DA 50-200mm lens, 2008

So, I use the 50-200mm lens primarily for shots at either extreme of its length, and I already have a 50mm fixed lens that takes better photos than the zoom at that distance. Moreover, breaking out just picks with this lens shows a three-to-one preference for 200mm than for 50mm. I think that means I need a long prime. Ka-ching!

I can also consider crop: How am I doing at composing in-camera? Here’s how often I crop, by lens, as a percentage:

	smc PENTAX-DA 18-55mm F3.5-5.6 AL   9.13 %
	smc PENTAX-DA 21mm F3.2 AL Limited 17.67 %
	smc PENTAX-DA 50-200mm F4-5.6 ED    6.93 %
	smc PENTAX-DA 70mm F2.4 Limited    23.78 %
	smc PENTAX-FA 35mm F2 AL           10.71 %
	smc PENTAX-FA 50mm F1.4            24.67 %

And, when I do crop, how much of the original composition do I keep?

	smc PENTAX-DA 18-55mm F3.5-5.6 AL  78.3 %                            
	smc PENTAX-DA 21mm F3.2 AL Limited 81.8 %                            
	smc PENTAX-DA 50-200mm F4-5.6 ED   81.6 %                            
	smc PENTAX-DA 70mm F2.4 Limited    80.9 %                            
	smc PENTAX-FA 35mm F2 AL           83.4 %                            
	smc PENTAX-FA 50mm F1.4            82.5 %

So, I’m cropping quite a bit. As Marc found in his exploration, these numbers go up when I filter by picks. I was surprised that I crop as much as I do with the DA21mm in particular, since I think of my use of it as being mostly for wide landscapes; but even those often enough are a bit crooked, enough to warrant at least some adjustment of tilt —- and Lightroom calls that adjustment a crop (fairly).

Does cropping mean I do a poor job at composing in-camera? Possibly. I have to admit that knowing I can crop gives me a conscientious freedom when I’m shooting, but these numbers give me something to think about. Maybe careful composition will be something to work on as I go forward.

We can cut all this in a few other ways. I’d like to take a look at my common keywords during a given time period, for example, but that will wait for the follow-up post, I think. This is more than enough nerdery for one January 1st afternoon.

Quick Lightroom 2 Tip: Show untagged images

I haven’t used Lightroom 2’s library filter extensively, but it’s really pretty powerful and I found a good use for it this morning. Having tagged a big batch of new images somewhat helter-skelterly, I found myself wondering if I could view just the images that I had not yet tagged, so that I could take care of them and be all done with this round of keywording. The library filter does this handily:

lightroom 2 metadata browser - setting presets

Open the filter pane (hit \ to activate and deactivate it), then select the “text” controller, and set it to “Keywords” “Are Empty”, and there you go: Lightroom 2 will display just the untagged images in the current view. Very handy, and the display will update as you proceed to tag, slipping newly-tagged images out of view.

I saved this library filter view as a preset using the dropdown controller available in the upper right of the pane, so I can quickly retrieve it any time I like.

Update: I see the following in a good post over at the O’Reilly Lightroom blog:

One of the example smart collections that is installed with Lightroom is Without Keywords. Take a look at the image count for this collection. If it’s anything other than 0 then get in there and add some keywords!

Nice! I haven’t much explored smart collections. That built-in collection is a great find.

Lightroom 2 Raw Development: Exposure

Last time in my ongoing ramblings on Lightroom 2 raw workflow, I focused on white balance. Today we move along a bit further with some elements of exposure.

A note about workflow seems pertinent: Lightroom’s preset abilities are pretty extensive, but I don’t make a whole lot of use of them, particularly for raw development (Though I do have lots of metadata presets that I apply at import to apply information about location). The develop presets make the most sense to me for repeated, reliable situations: Shooting in a studio, for example, where you know combinations of lens and light and in which case the effort to tune and save those presets will be rewarded. That’s my experience so far, so I’ve neglected to spend much time working with develop presets.

So let’s talk about exposure. Lightroom provides an exposure histogram in the upper right pane. The histogram quantifies the amount of light found between black, at the left end, and white, at the right side of the plot. A histogram with nothing but a steep peak at the right end will likely appear washed out or overexposed; likewise, a plot with nothing but a peak at the far left will be dark, lacking highlights.

Lightroom 2 histogram screenshot

Here’s what this particular histogram is telling us:

  • In its histogram display, Lightroom provides some basic image information: ISO, focal length, aperture, and shutter speed.
  • We have light basically all throughout the plot’s range, with a particular peak close to the bright end.
  • Overall, the exposure of this image isn’t bad. We can adjust exposure either with the exposure slider or by grabbing and dragging the center portion of the histogram left or right to obtain the kind of effect we want to get. I tend to boost exposure a bit when working up my photos —- but be careful when sliding exposure rightward: too much will add speckly noise to the image.
  • The filled triangles at the upper left and right corners are clipping indicators; since they’re filled, it means that this image has some amount of both cut off highlights (right end of histogram: “blown highlights”) and clipped darks (left end of histogram). That is, some detail is lost in the brightest and darkest areas of the photos. The more clipping, the brighter the fill of the triangles. You can see the clipping in the histogram as the blue peak pressed up against the left edge, and the grey/red smooshed into the right end. To see just what’s lost, hit J. The blue and red superimposed over the image represent lost darks and lights, respectively.

Here’s a sample image. On the right is an illustration of the highlight clipping revealed by hitting J in the develop module:

Lightroom 2 sample image Lightroom 2 sample image

To bring a bit more detail back into this image, especially some of the texture in that yukon gold, you use the “recover” slider in the develop pane, or simply drag the rightmost edge of the histogram back toward the left side: hover the mouse along different parts of the histogram to identify the various draggable regions. As you increase the “recovery” value or drag that right end leftward, you’ll see the red blown highlights indications start to fade, as in the image below where the recovery slider has been pegged all the way rightward:

Lightroom 2 sample image Lightroom 2 sample image

Adjust per taste. See how much more gray the background of this image has become by pegging the recovery? Lightroom can often recover a ton of detail in highlights, but at the extreme end the cost will be a dimming of the overall image.

The same goes for darks: You can bring detail back out of a black swatch of image by reducing the “blacks” slider or by dragging the leftmost end of the histogram back to the right a bit. I often find it works nicely to drag that histogram until the leftmost slope of the dark end of the plot rests just at the edge of the display — that is, where you’ve just barely recovered all the clipped dark areas.

While measurements of light and dark can be quantified in the histogram, appreciation of light and dark are subjective. Leaving blown highlights (or intentionally blowing them by increasing the exposure — grab the middle of that histogram and pull it rightwards until you see something you like) and lost black regions of an image may help convey exactly what you want an image to do.

One more exposure-related tip: Fill light. Use this slider (or, as always, slide the histogram itself — the fill light region is just shy of the leftmost “darks” region of the plot) to bring up the light in the image’s shadowy regions without boosting the overall exposure. This can do wonders for errant shadows across a face.

Be watchful for new noise when you add fill light, and it’s a good idea to bump up the “clarity“ slider in conjunction with fill light, since fill light tends to take a bit of the punch out of the image.

So, there’s a quick take on adjusting exposure with Lightroom 2. Between white balance and exposure, you have the core set of adjustments to your raw images covered, but there’s more to come: Further tuning involves sharpening, noise reduction, and crops, as well as other adjustments we might make to color. Some discussion of those are coming up in my next installment of Lightroom for the Rest of Us (or whatever it is I’ll be calling this series by then).

From Raw to Rawwwr: Lightroom 2 Raw Development for the Rest of Us

I have a casual obsession with Lightroom 2 and have had a load of fun tinkering with the Lightroom catalog database to extract information about my photographs. More recently, I’ve been writing over the past handful of weeks about my workflow in Lightroom 2 from the perspective of an enthusiastic but non-professional user. I’ve previously covered organization and keywords. This time around, the topic is using LR2 to work up raw photos.

Why would one shoot in raw? Raw is an image format meant to preserve as much original image data as possible; that information can be manipulated much more flexibly and effectively than the comparably limited range of image data contained in a JPG or TIFF image. A raw image with bad exposure or white balance can be salvaged, while a JPG — lacking the image information of raw — offers far less latitude for recovery. Meanwhile, JPGs produced by most digital cameras have some amount of sharpening, exposure, and contrast adjustment applied in-camera, while raw images are essentially straight representations of what the camera “saw” when taking the photo. This means that raw images will almost always need a bit of color tweaking and sharpening. Hence the eternal quest for a reliable, easy-to-apply workflow for “developing” raw images to best effect, and the subject that brings me to the MacBook today.

After importing and doing preliminary tagging, flagging and deleting of images, I start in on working up the raw images.

White Balance

White Balance is the first thing to tackle. The point of correcting white balance is to give the right color tone to the “neutral” (white or grey) colors in your image. In other words, you want white to look white. If the WB is off, your photos will have an unnaturally cool or warm cast.

In Lightroom 2, you can adjust white balance in both develop and library modules using presets for different light types, and in the develop module via the temperature sliders. (There are a couple of quick-develop buttons for temperature, but I generally avoid using them.)

The most common white balance adjustment I make starts with selecting the “tungsten” preset. This immediately removes the too-hot feel of photos taken in incandescent light — ie, most sources of indoor, non-flash light.

LR2 screenshot

After using the Tungsten preset, it’s usually necessary to make a few adjustments: This is when I use the Tint and Temperature sliders in the develop module for subtle shifts, often to add just a bit more warmth to the light by nudging the Temp control a tiny bit back to the right. It’s a good idea to swing the Temp control around a bit just to see what kind of difference it makes to give a bit more warmth or coolness to an image.

Mixed light provides a dilemma for white balance. Light from an overhead lamp plus natural sunlight through a window, for example, can be tricky to adjust. I find that setting the Tungsten preset will cool off the overall color cast, but will also amplify the parts of the image that are already a bit cooler — the natural sunlight — giving them a sharp blue tone, especially around highlights. My solution to this is to desaturate the blues a bit, which takes the edge off that coolness. In Lightroom 2, you have even more ability to make that correction by using the adjustment brush to spot-desaturate the highlights that are made too cold by adjusting to Tungsten. The below shot illustrates an image that’s been tweaked by first presetting WB to Tungsten, and then backing off the blue saturation just a few points.

LR2 screenshot

But wait, there’s more! While the Tungsten preset is good, it’s not always quite right. As alternative, use the White Balance Dropper — that’s the dropper button next to the WB selector in the develop module. Click it, then click a spot in your image that should be nice and neutral, a white or grey spot. Presto! Instant white balance. When I can’t quite decide on the right balance, I often use this tool on various spots of the image to quickly try out a range of white balances. The shortcut for the WB dropper is W, which you can use in both grid and develop modes.

Once you’re happy with the white balance of a given image, you can quickly apply those settings to the remainder of similar images with the “sync settings” command: Swap over to grid mode, select the corrected image first, and then click-select the others with similar light, and finally sync ( shift-cmd-S).

There’s much more to the raw workflow: Exposure, sharpening, and so on, and I’ll tackle some of those next time.

My Lightroom 2 Backup Strategy

Related: A more recent post about archive and backup in Lightroom.

With this morning’s comment from martie asking about a crashed hard drive, I got to thinking about making my own Lightroom 2 backup plan a bit more automated and reliable. My general approach is to periodically copy my catalog file and image directories to an external hard drive, but there’s been nothing systematic about it until now.

I’ve previously described a bit of my Lightroom file structure, noting that I import new photos into a single directory per import. As part of a strategy to save space on the MacBook where I do my actual work, I periodically move those folders to an external hard disk currently named Grundle. This is simply a matter of dragging the folder, in the left-hand directories pane of Lightroom, from one hard drive to another.

Lightroom display of multiple drives

While this copying step is manual, the rest of the system is now automated, thanks to this tutorial at MacResearch and a bash script by Aidan Clark. The bash script took just a bit of tinkering to work with Lightroom’s catalog file, which by default will have rsync-breaking spaces in it, and to perform the second backup from the external volume to the iMac. I’ll post those specific and very minor modifications if there is interest.

Here’s the final result: Using OS X’s launchd tool, whenever I mount Grundle on my MacBook, whether via network or direct firewire connection, my Lightroom 2 catalog file is copied to Grundle using rsync. And, whenever I mount Grundle on the upstairs iMac, a similar combination of launchd and rsync copies both the catalog file and the image directories from Grundle to the iMac. This means that in the course of regular use of my two Macs and that external drive, both my Lightroom catalog and folders full of images get backed up.

One caveat to this system is that the backup of the image folders still involves that manual step of moving them from the laptop to Grundle. I could automate this the same way the catalog backup is done, but that could potentially mean trying to backup a gig or more at a time over the wifi network — a time- and bandwidth- consuming process that isn’t really necessary. Now, the obvious down-side is that the newest photos I’ve taken are always the ones most vulnerable to data loss, and that’s obviously not a highly desirable thing. But I’m satisfied with my current workflow of moving folders to Grundle generally when I’m done working with that set of images. I’ll continue to think about this situation and may come up with some additional redundancy for that stage of processing.

Update: Okay, I buckled. A bit more tinkering and I now have my current folders of raw images copied to Grundle. After I relocate the folder using Lightroom, the folder will disappear from the backup directory, so I don’t have redundant backup files stacking up anywhere. Nice and clean, and everything’s safe.

Update the second: One item I neglected to mention in the original post was the automated backup feature built in to Lightroom: Available in the catalog settings menu (alt-cmd-,) this feature performs scheduled backups of your catalog file only, to a location you specify, and can be set to run a backup on any of several schedules. My process above includes allowing that backup to run weekly — it never hurts to have a little more failsafe security. The benefit of automating getting that backup to another hard drive is one more important layer of keeping safe your data.

Dealing with duplicate keywords in Lightroom 2

I’ve noted recently how slick it is to be able to use nested keywords in Lightroom: It’s a piece of cake to select a set of photos, hit cmd-K, and enter “mount humphries > mountains” to assign the “mount humphries” as a child keyword of “mountains”.

However, as noted by a poster to a thread about keywording over on the Flickr Lightroom group, this creates a potential problem: If the child keyword is already used, Lightroom 2 will end up creating a duplicate keyword; you’ll end up with one “mount humphries” without a parent, and one “mount humphries” keyword with the parent of “mountains.” So, by trying to be hierarchical with your keywording, you’ve actually splintered your keywords. Not helpful!

Having already found a fun way to explore relationships between and frequencies of my keywords, it occurred to me that I might have some ready-made tools to help with this situation: The need to find and deal with duplicate keywords.

As this article became more popular, I worked through a couple of alternative methods and organized things a bit further. To date, I describe three methods of identifying duplicate keywords:

  • Full auto: Requires some scripting but is the most expedient way to go about it (and my favorite).
  • Semi-auto: Requires the awk tool to identify duplicates but doesn’t rely on any sqlite3 code to pull from the LR database.
  • Full manual: Uses LR’s built-in export tool and MS Excel to get what you want. Lots of steps, but it works.

After finding your preferred method, read on to see what to do with all the duplicates you identify.

The full auto/scripted method

Thanks to Dieter for putting me on to a much quicker, straightforward way to identify duplicates using SQLite only — no awk:

select count(name) as num, name from AgLibraryKeyword group by name having num > 1

I’ve kept the original SQLite+awk method below for posterity:

First I needed to find any duplicates created — potentially — by the ad-hoc nesting of keywords. A quick modification to my keyword frequency script produces a list of dupes:

# display a list of duplicate keywords in Lightroom 2 cp ~/Pictures/Lightroom/Lightroom\ 2\ Catalog.lrcat ~/lightroom.lrdb /usr/local/bin/sqlite3 -csv ~/lightroom.lrdb 'select ImageCountCache, name from AgLibraryKeyword;' > /Users/alan/lr-keywords.csv awk -F , '{print $2}' lr-keywords.csv | sort -n | uniq -d rm ~/lightroom.lrdb rm ~/lr-keywords.csv

As with the frequencies, this short script makes a backup copy of my Lightroom 2 database, then calls on sqlite3 to extract the list of keywords; the difference is in the 4th line, where I use awk to pull out the tags and then pipe them through two built-in unix functions to print a list of any duplicates. In my case, it yielded the following:

Running the duplicate keyword script

Perfect! A list of keywords that appear to be duplicates.

Note: The script above works with OS X Tiger and requires an upgrade to the default version sqlite. It ought to work with Leopard as long as sqlite is present. Windows? Don’t know; either via cygwin or separate binaries, awk and uniq should be available for Windows, and there is a sqlite for Windows download at the above link.

Skipping the sqlite step

Update 17/Dec/2008: If you’re not eager to delve into sqlite, you can still make this work, but you’ll still need to have the awk tool. OS X users, you should be good to go, since awk comes with the OS; Windows users, you can download awk for Windows. First, manually export your keywords list from your catalog: Metadata > Export Keywords…, and save the file as lrkeywords.txt, and then run the following one-liner script from a shell/terminal:

awk 'BEGIN {FS = " "}; {$2 = $2; if (match($0, /{/)0) print}’ lrkeywords.txt | sort -n | uniq -d

Just as in the sqlite version, this one-liner parses your keywords file and returns the list of keywords that appear more than once and are not identified as synonyms. You can then reconcile the duplicates as described above. I prefer the single-step version that extracts directly from the database, but hope this is useful to a few folks.

The Full Manual Process

If you’re averse to both the sql steps and to using awk, you can use MS Excel to identify the duplicates. I think it’s far more cumbersome than either of the above processes, but it works. Here goes:

Manually export your keywords from Lightroom: Metadata > Export Keywords ….

Open that export file in Excel: Open as a text delimited file, but uncheck all the delimiters; you don’t want excel to parse along spaces or tabs, since both of those characters appear in the file but not as record separators.

You should see a single column of keywords something like this:

screenshot

We need to slightly clean up that column to remove leading whitespace. We’ll use the CLEAN function to do that: In the column next to your keywords column, enter the formula =clean(a1), and then drag that formula all the way down the keywords column.

screenshot

In col B, you now have a whitespace-trimmed set of keywords, but because of the clean() formula, you can’t manipulate it further. Select that entire column, copy it, and then use paste special to paste the column as values into column C.

screenshot - cleaning up keywords list with paste special

Now we’re ready for the final steps: Sort, flag, and filter. Select column C, then go to Data > Sort, and sort ascending by column C. Now, in column D, drag the following formula from the first row to the last: =IF(C2=C1, "!", "").

screenshot - applying the duplicate flag formula to the sorted list

That just fills the column with a flag if column C has a duplicate. It’s a low-budget search, but it works as long as the list of keywords is sorted. Finally, use Data > Filter > Autofilter, and click-select the ! in column D. You’ll now have a filtered list of duplicates from your original keyword list, which you can resolve as described above. Note that you’ll have a number of keywords surrounded by {} or [] brackets; these are keywords entered as synonyms or categories, and you should ignore them when you are addressing duplicates.

screenshot - the list of filtered duplicate keywords

You can see from my current duplicate list that I’ve been working heavily on food-related keywords as we cruise through the holiday season.

All told, that whole manual process should just take a couple of minutes once you have the steps sorted out. Because the steps are manual, it’s not as easily-repeatable as the automated sqlite+awk approach, but it does work. I hope someone finds it useful!

Dealing with the duplicates

Whatever method you’ve employed, at this point you have a list now — let’s check out if it means what I think it means. Switching back over to Lightroom, I can filter for all photos with the “mount humphries” keyword:

Lightroom listing duplicate keywords

Sure enough, I have 12 images tagged with “mount humphries”, and 11 images with the same tag set as a child of “mountains” (as an aside, I see that I have well over a hundred images with the “mountains” tag that could probably use some more granular tagging).

My first impulse was to try to just drag the non-child “mount humphries” into the “mountains” tag; this works, after all, with other keywords. But in this case, it won’t do the trick, presumably because there is already a “mount humphries” keyword there — Lightroom won’t let me add a same-named child.

To reassign the keyword to the parent, you need to take a few more steps: First, click the right-pointed arrow to the right side of the duplicate, non-child keyword; this will navigate to all images assigned that keyword. Then, in the grid view, simply select all (cmd-A), and then check the child keyword to add it to all of the selected images (the checkbox is to the left of the keyword, and appears when the mouse cursor hovers over the keyword), and you’ll see the count increase accordingly. Next, un-check the duplicate, non-child keyword in the keywords panel. You’ll see its count drop to 0. The order of those two check-uncheck steps is important: If you uncheck the non-child keyword first, you’ll end up with an empty selection and nothing to apply the proper keyword to.

Lightroom displaying duplicate keywords unchecked

There! My “mount humphries” keyword as a child of “mountains” is now assigned to all 23 original images, and I can delete the duplicate, non-child tag.

So, with an approach like this, ad-hoc keyword nesting shouldn’t be feared: We can identify duplicates created by nesting, and, in a matter of seconds, apply the same nesting to any previously-tagged images. And, once you’ve resolved the duplicate, any further assignment of the focal keyword will always assign it, appropriately, as a nested tag. Pretty slick, I do say.

A Lightroom 2 workflow for the rest of us

(Welcome to readers coming via the nice link from the Lightroom team at Facebook! Consider checking out the rest of my Lightroom writing.)

—-

While Lightroom gets a lot of love from professionals who take a thousand photos at an event and need to manage a paying production workflow, it’s also great for enthusiast-level users like me: I usually work on the scale of a weekend’s or a trip’s worth of photos that need organization, a place to live, some pruning and keywords, and whatever sorts of RAW workup that’s necessary. Last week I wrote up a few general Lightroom tips with some vague intention of documenting a practical workflow; here are two high-level elements of my workflow: Import/File structure and metadata/keywords.

Import and file structure: I shoot in RAW and use Lightroom to import photos directly from the SD card via USB card-reader. Lightroom presents a number of options for storing those imported files: In the past I put imported photos in by-day folders, meaning one directory for each file date. So, for a handful of days’ worth of photos I’d have a hierarchy of directories like 2008-09-01, 2008-09-02, and 2008-09-03. I’ve recently switched to a storage option that, so far, I really prefer: I import photos to a single directory based on import date rather than file date, with a descriptive suffix to that directory. The nice upside to this approach is that a weeks’ worth of photos — a five-day trip somewhere, for example — all end up in a single directory, and can all be viewed and processed as a set (without having to put them all into a collection). Since Lightroom is so handy with metadata, I don’t lose anything by stepping up in granularity from by-day storage to by-import storage. Additionally, having fewer directories hanging around makes it easier both to backup and to browse my photo file tree with other applications.

Below is a screenshot to illustrate the import and directory naming stage:

Lightroom Import Dialog

On import, I assign a metadata preset and whatever keywords are applicable to the entire import set of photos.

After import is a first pass to identify picks and rejects and assign keywords, which I described last time around: Set grid mode to a large-ish preview size, and use shift-P and shift-X in grid mode to mark any clear favorites and rejects. After that pass, it’s keywording/metadata time.

  • Metadata: Lightroom has two-dozen or so non-keyword metadata fields that, frankly, I don’t have much use for. I tend to use only a few of them (in the “Image” metadata category) to assign geographic location of photos taken: I have metadata presets for common locations, and I assign the rest on an ad-hoc basis. At export time, some of this metadata is included as keywords, so my exported photos end up with, at least, city and state information attached.
  • Keywords: You can keyword forever in Lightroom. I start keywording with a single keyword set that broadly covers my most common uses (The “edit keyword set” function is accessed in the Keyword Set menu in the right-hand Keywording panel): I call this one “Common/Home”, and it contains, among other keywords, “Heather”, “dogs”, “flickr”, and “food” tags. I know the order of these tags and can quickly make assignments with alt-num combinations. With my “Common/Home” set active, alt-7 tags a photo with flickr, alt-8 with Heather, for example. Wondering what your common keywords are? Try exploring your database for common lightroom keywords, and take advantage of nested keywords and keyword hierarchies for the greatest efficiency in assigning keywords.
  • In Lightroom 2, the related keywords function adds a whole new dimension to commonly-used tags. Once you’ve assigned one or two from a preset menu (as above), you’ll get a set of keyword suggestions, as well. Lightroom 2 really does facilitate both identifying and applying keywords in a useful way.
  • On applying keywords: As with metadata, I’ve found it most useful to reduce the size of previews in the grid window and apply keywords in groups to a selected set of images first (as many as possible; ctrl-click (shift-click for a range) or shift-arrow with keyboard to select multiples), and then step through photos one-by-one, adding additional keywords as needed.
  • Lightroom lets you display keywords for a given image in a number of ways. I like to keep the keyword display set to “will export” — That’s generally the most complete display of the tags assigned to a given image, including synonyms and parents.

Possible items for next time? Super-basic RAW development, exporting, and collections: See my intro to RAW development!

Photography: Hitchhiker's Guide to Lightroom

I spend a lot of time in Lightroom 2 these days. I’m nobody’s pro, but I shoot a lot of photos, and after having used Lightroom (and now Lightroom 2) for a while now, I think I have a pretty good, simple, enthusiast-style workflow sorted out. I’ll summarize the workflow itself (importing through working up images) in follow-up post. Here are a few general tips that seem to work well for me:

Essentials, or Stuff I use constantly: I use Picks and keywords extensively. Reviewing newly-imported photos, I mark anything that I like right off the bat as a Pick by simply hitting shift-P as I scan through the gallery (and shift-X to immediately mark others as rejects; the shift modifier will mark the current photo and move on to the next shot). As I revisit a set of shots later, I find myself repeating this process; while those subsequent passes primarily identify further Rejects, I do occasionally find more Picks after starting to work up other photos. After each pass through a gallery, I use cmd-DELETE to remove (and delete) all the Rejects.

This has been a nice insight for my process: It means that I am fairly conservative when it comes to Rejects. That is, I don’t mark as Rejected 1) unless a photo is obviously bad (bad focus, blur, composition I really dislike, etc.) OR 2) until I’ve spent some time on photos in a set that I do like right from the get-go. This frequently helps give me a sense for appealing qualities of photos that I might not have noticed or thought of initially.

With a gallery through at least a first pass of identifying Picks and Rejects, I apply keywords. As with many aspects of processing photos, Lightroom has lots of ways to do this. There’s a jobber called Keyword Painting that I don’t use, because it’s always been much faster to simply select sets of photos and then apply keywords to the selection. In Lightroom 2, cmd-K focuses on the keyword entry box, which will auto-complete as you type. Lightroom 2 also has “recommended keywords” functionality, so that as keywords are assigned to a photo or set of photos, a new set of co-occurring keywords is identified and displayed for easy additional assignment.

Although I like to use a large-ish image preview (hit = to increase the size of preview images in the gallery grid) for screening for Picks and Rejects, for keywords I like to shrink the grid size (keyboard shortcut -). This fits more images into the grid and allows me to select larger sets for group assignment of keywords.

Lightroom allows for keywords to be nested, and there’s a great shortcut for accomplishing this: When entering keywords, separate child from parent keywords with a > sign: flickers > birds, for example, or burgers > food.

Simplifying, or Things I don’t use in Lightroom: Beyond keywording, Lightroom has at least a trio of way to identify and categorize photos: You can flag photos as Picks, label them with colors, and rate them with zero through five stars. I don’t use colors or stars at all. They may be highly useful for some situations, but they just clutter the cognitive space where I think about my photos: “Is this a three-star green photo, or a four-star blue one?” So except in the rarest circumstances, I haven’t yet found a use for ratings and color labels.

Indispensable keyboard shortcuts: There are grundles of these, but the shortcuts I use all the time are:

  • G, E, D: Gallery, Editor, and Develop modes
  • P: Mark as Pick (modify with shift)
  • X: Mark as Reject (modify with shift)
  • cmd-delete: Remove Rejects (optionally delete from disk)
  • cmd-K: Assign keywords
  • W: Jump to White Balance selector in Develop mode
  • R: Crop tool in Develop mode
  • J: Show clipped darks and highlights (Developer only; in gallery, changes display of thumbnails)
  • L: Cycle the lights (view on black)
  • tab/shift-tab: reveal/hide menu panels

Next time: The library filter, file organization, workflow, and Lightroom+Flickr?

Frames and borders with Lightroom 2

Update 7/26/2009: There are some fantastic new features of LR2/Mogrify to check out:

  • Relative-sized borders (which I’ve written about before) are super.
  • As is relative-sized annotation text. No more calculating your scaled text size to apply to an export, since you can set the size to a percentage of image width or height. Very cool. (This goes for watermarks, too)
  • More border features! A checkbox for “identical borders” makes it much easier to set uniform borders. And inner borders are now supported for the creation of an inset frame with variable opacity.
  • Setting compression by file size: Specify a file size and LR2/Mogrify will compress the image accordingly so as not to exceed that file size.

Give these a try and don’t forget to donate to Timothy’s work if you find it of use.

Also, note that you can use Lightroom’s post-crop vignette feature to generate curved borders (in black or white) with less flexibility but also without plugins.

Update 12/21/2008: Much of this writeup now has more historical than practical value, since Timothy Armes has updated LR2/Mogrify to support multiple border options within the plugin’s own control panel. You can specify different-sized frames & borders without any extra monkeying around. Nice work, Tim!

A question recently came up in one of the Lightroom groups over on flickr about creating images with large borders on just one side — space within a frame to place a title, for example, but just along a single edge of an image. The poster wanted to create images such as those found here, and wondered if it was doable without diving out to an external tool like Photoshop. The first working proposal was to use a graphical frame applied in the print module, but that isn’t an ideal solution for me; it still requires setting up that frame with something like Photoshop, and to apply it you have to switch modes. So I tinkered a while with a photo I took a couple of nights ago, and managed to get what I think is a nice solution via a direct export from Lightroom using the fantastic LR2/Mogrify plugin from Timothy Armes.

The out-of-the-box options for this plugin don’t provide a capability to create different-sized borders to an image, but the underlying engine for the plugin, Imagemagick’s mogrify tool, does — after a fashion. So, in a nutshell, the trick is to use the command line element of LR2/Mogrify in addition to its other features, to add to the picture’s canvas size before performing the other operations.

Monte Vista neon

There are just a couple of tricks to get this to work smoothly using LR2/Mogrify. First is to add the extent command to the mogrify configuation, specifying the resulting size of the image you want to export:

I’ve specified the command -background white -extent 3008×2158 to be prepended to the mogrify command line that LR2/Mogrify will execute for me. I’m exporting an original image that’s 3008×2008, so I’ve specified 2008×2158 to the final image — adding 150 pixels, which will be filled with a white background. Next I use the built-in features of LR2/Mogrify to add the colored frames and the text overlay.

Because the extent command was applied to the beginning of the command line, the borders will be applied to the new image — the one with the bigger lower border created by extent.

The text overlay is just a bit strange. Note that instead of specifying the text to fall at the bottom, I’ve placed it at the top center of the image, with an offset of 2158 pixels. For some reason, directly placing it at the bottom center reverses the position of the new white border — it ends up at the top of the image, through some kink of mogrify that I can’t quite sort. It’s easy enough to compensate with the offset.

Export away, and that’s all it takes. You’ve built an image with a nice broad frame and caption, all right from Lightroom’s export panel. No Photoshop or print module necessary. Fun.

Lightroom 2: Related Keywords are Dreamy

As it happens, Lightroom 2.0 has just the thing I daydreamed about a handful of months ago. The new version’s data includes a table of keyword co-occurrences that makes it possible to produce things like this:

My flickr tag neighborhood

This graph shows keyword relationships that occur within a hop from my “flickr” keyword — which I use to keep track of photos that I upload there. In other words, it’s sort of a descriptive keyword neighborhood of what I’ve put up on flickr.

Color is a little subjective. The darker blue, the higher the ratio between unique neighbors and total neighbors. That is, darker blue nodes are connected to relatively few unique other neighbors than the lighter blue nodes.

Of course, you could use any focal keyword for this kind of thing: Starting with a lens-specific keyword would produce a rough map of the neighborhood associated with that lens, and might reveal how I tend to use that lens. The possibilities are pretty endless — and totally a fun kick in the pants to tinker with.

Keywords in Lightroom

[ Interested in my keywording workflow or duplicate keywords posts? ]

 

The last time I wrote about Lightroom, I was using sqlite to pull out frequencies of focal length. This time it’s keywords: Lightroom lets you build any number of custom keyword sets to apply to photos. It automatically builds a set of “recently used” keywords, but I thought it would also be handy to have a set of my most commonly-used keywords. While Lightroom has a command to export a list of keywords, that list doesn’t include frequencies. Keywords are stored in Lightroom in a table called AgLibraryTag (AgLibraryKeyword in Lightroom 2, see update below). Conveniently, Lightroom writes a count of each keyword to the same table, so it’s easy to get out all the information we need. (Note: The frequency in this table is a cached value and may not reflect the up-to-the-minute reality within your database. Rather than constantly update its databases Lightroom seems to update this count when you view the discrete keywords. I’m not sure how to force a library-wide update of all keyword counts. This is probably close enough, and is simpler/quicker than counting keywords image-by-image.)

Rather than run this data through R to build a histogram as I did with the focal length data, I just use awk this time to make a list with the most frequently-used tags at the bottom. With this list, you can easily build a corresponding tag set in Lightroom.

Remember to change paths to suit, and that (on OSX) you’ll probably need to upgrade your version of sqlite for all this to work. Also, always always work from a copy of your database as this script does.

# display a sorted list of lightroom keywords (Lightroom 1 only) cp ~/Pictures/Lightroom/Lightroom\ Catalog.lrcat ~/lightroom.lrdb /usr/local/bin/sqlite3 -csv ~/lightroom.lrdb 'select ImageCountCache, name from AgLibraryTag where kindName="AgKeywordTagKind";' > /Users/alan/lr-keywords.csv awk -F , '{print $1" "$2}' lr-keywords.csv | sort -n rm ~/lightroom.lrdb rm ~/lr-keywords.csv

Daydream: A map of how keywords relate to one another would be awesome.

Update: The above daydream is now possible with Lightroom 2’s related keywords functionality. Write-up here.

Update again (Oct 4 2008): The old code didn’t work with Lightroom 2 due to some things moving around in the database. The below seems to fix it and obtain keyword frequencies for LR2:

# display a sorted list of lightroom keywords # Updated for Lightroom 2 cp ~/Pictures/Lightroom/Lightroom\ 2\ Catalog.lrcat ~/lightroom.lrdb /usr/local/bin/sqlite3 -csv ~/lightroom.lrdb 'select ImageCountCache, name from AgLibraryKeyword;' > /Users/alan/lr-keywords.csv awk -F , '{print $1" "$2}' lr-keywords.csv | sort -n rm ~/lightroom.lrdb rm ~/lr-keywords.csv

Trains composite

Trains composite

I love the discoverability of Lightroom. First you find how neat it works to just manage lots of photos — and that process gets deeper as you use it more — and then you get better at processing raw images, and then you start to explore the other modules and see how the enable you to do creative things all in a single package.

Photo tool nrrdery

Update 1/1/2009: I’ve built some much more sophisticated sets of stats from my Lightroom 2 catalog.

Update 6/27/2007: Lightroom v1.1 is out (it’s real, and it’s spectacular, see the O’Reilly Lightroom Blog for more), and it changes the database from a “library” to a “catalog.” In terms of this little tool, this change seems to only entail changing the filename referred to in the wrapper shell script — as I’ve done, below. Otherwise, generating a focal length histogram seems to work just as it did previously.

Camera Nrrdery

For fun: You can see that I use my fixed 50mm and 21mm lenses far more than anything else I’ve got. That’s because they’re so very pretty.

I use Adobe’s Lightroom to manage my RAW photos. It’s a wonderful, splendid tool. Among its features, it provides a handy metadata browser of your photo library, and includes the ability to browse by lens. Recently James Duncan Davidson mentioned being interested in plotting his use of various focal lengths, and commenters responded with a number of good solutions. Since Lightroom uses a SQLite database for its library, tools like SQLite Browser can be used to scan through the database file itself and export tables, at which point it’s straightforward to grep and find focal lengths. This is pretty slick all by itself, but I thought I’d put together a quick tool to automate the extraction and generation of this data. To do that, I use sqlite3 from the command line to dump the metadata table to a file, and then a short bit of R code finds the focal lengths and builds the histogram. The sqlite3 commands and the R code are invoked via a shell script that makes a copy of the main database to work with and cleans up the temp file when it’s all done.

Howto

If you made this this far, you might actually be interested in how it’s all done. After some tinkering, I found from Jeffrey Friedl’s Blog that Lightroom’s current database needs a newer version of sqlite3 than that which ships with OSX. With that update installed, sqlite3 will handle your Lightroom database without any problems.

Here’s the shell wrapper. Change paths to suit:

#!/bin/bash cp ~/Pictures/Lightroom/Lightroom\ Catalog.lrcat ~/lightroom.lrdb /usr/local/bin/sqlite3 -csv ~/lightroom.lrdb 'select xmp from Adobe_AdditionalMetadata;' > /Users/alan/lr-metadata.csv R CMD BATCH /Users/alan/bin/lr-getfocallengths.R rm ~/lightroom.lrdb rm ~/lr-metadata.csv convert ~/lr-focallengths.pdf ~/lr-focallengths.jpg

And here’s the R code, which lives in lr-getfocallengths.R and is called by the shell script. Again, fix paths for your own circumstances:

lr <- file("/Users/alan/lr-metadata.csv", "r") lrlines <- readLines(lr) temp <- gsub("(/1)", "", lrlines[grep("exif:FocalLength>", lrlines)]) lengths <- as.numeric(gsub("([^[:digit:]])", "", temp)) lengths<-lengths[lengths<=1000] pdf("/Users/alan/lr-focallengths.pdf") hist(lengths, main="Histogram of Focal Length Use",    xlab="Focal length (mm)", ylab="Number", breaks=seq(0,200, by=4)) dev.off()

A few things to note:

  • Depending on how your version of R is compiled, you can use jpeg(…) instead of pdf(…) to make the output file. My R isn’t currently compiled with jpg support, so I build a pdf file and then use convert on it.
  • There’s some noise in the metadata that leads to the erroneous identification of focal lengths like 83456000. That’s not right at all. I skim off everything above 1000 in line 5 of the R code. (Which is still sort of silly. My longest lens is presently 200mm.)
  • Relatedly, the x axis of the histogram only goes up to 200. To change that, modify the seq(0,200, by=4) accordingly — you can change the upper bound as well as the width of the bins.
  • A really slick way to do all this would be to properly parse the exported table in order to combine data, in order to limit the data to, for example, “favorites” by focal length. These aren’t in their own fields in the database, however, but rather all within a single column that holds all an image’s metadata, which makes it harder to select on multiple conditions. That’s a trick for another day.


About, the short version

I’m a sociologist-errant. This site is powered by Textpattern, Kaizen Garden and the sociological imagination. For more about me and this site, see the long version.

Copyright and so forth: Commenters own their own posts, and linked or excerpted material is subject to whatever copyright covers the original. Everything else here is mine, rights reserved.

RSS feed