schussman.com logo

Another year of photo data!

Following in the modest two-year tradition I’ve established (see 2007 and 2008 posts), here is my 2009 photo data from my Lightroom catalog!

[ quick howto: Lightroom 1 & 2 (and 3) databases are in sqlite3 format, which means that freely-available tools can extract data from them. I use sqlite3, some shell scripting, and R (and occasionally excel) to produce summaries of that data. Why? Data offers some insight into the kinds of photos I take. Mostly, though, it’s fun. I’d be happy to expand on the actual code that goes into these plots, if there’s interest. ]

Below is a set of plots that summarize some of this year’s data. Click through to flickr to see the larger version.

2009 photo data!

What’s interesting this year? Well, crop ratios looked pretty similar to last year, so this year, for the first time (suggested in a post by Keitha, whose photos I admire tremendously, and whose Pentax lens set I envy with the fire of a million anti-glare-coated nine-aperture-bladed all-metal suns) I pulled out some information about aperture for each of the prime lenses that I shoot with. You can see these four frequency plots (for each of the Pentax DA 70mm F2.4 ltd, FA 50mm F1.4, FA 35mm F2.0 and DA 21mm F3.2 ltd lenses) in the left hand column of the image. Right off the bat you can see that I shot a lot with the FA 35mm this year (which is confirmed by the “overall lens use” plot on the right column). In fact, I took that lens along as my sole lens on a few long weekend trips to Ventura, CA, and the San Juan Islands, and really loved its performance. It does great at large apertures, but I also used it a lot for street shooting at f/8 and smaller apertures.

Runner-up in frequency this year is the FA 50mm F/1.4, which ordinarily I would say is my favorite lens (and it very much still is; it just wasn’t as convenient a focal length to take as my only lens on those vacations). Its sweet spot [where it’s sharpest but still has a nice narrow depth of field] is about F/4, which is where I primarily use it.

Neither the DA 70mm F/2.4 or the DA 21mm F/3.2 got as much use this year, but I really love some of the photos I took with those lenses. In fact, I carried these two lenses specifically for their light weight and trim size on the Flagstaff photowalk I organized in July.

Car / Cat Ranch house / wide Crow Pomegranite Backside Doorman

How did 2009 stack up to 2008? In terms of absolute frequency, nearly identical! I kept 1308 frames last year, compared to keeping 1340 in 2008. Far fewer of those are picks, or posted to flickr — though a good number are waiting for me to come back to, to finish workup or to make a print.

And that’s it for the 2009 photo stats! I did re-work my keyword network code, so perhaps can follow up this post with a little more about keyword relationships.

If you’d like to know more about extracting and summarizing info from your own Lightroom catalog, please let me know (and check out my other lightroom-related posts)

And, as last year, I hope soon to follow up with a report on my 2009 photo goals, and to set a few for 2010.

Today's Lightroom quick tip: Targeted adjustment shortcuts

It’s nice to keep learning things about Lightroom. It tends to have solutions for problems long before I realize that I need them, and so I was happy to find this morning that it has handy shortcuts for all the targeted adjustment brushes.

  • shift-alt-cmd-N: None (cancel adjustment)
  • shift-alt-cmd-T: Tone adjustment
  • shift-alt-cmd-H: Hue adjustment
  • shift-alt-cmd-S: Saturation adjustment
  • shift-alt-cmd-L: Luminance adjustment
  • shift-alt-cmd-G: Grayscale mix

Activating any shortcut will jump your develop panel to the appropriate controller, so you can keep your cursor focused on the image and still see the effects of your adjustments in the sliders or levels. It’s a nice, efficient way to minimize extra mouse movement and make quick modifications.

 

Check out the rest of my Lightroom posts for much, much more, including more database tinkering, keywording, and workflow.

Lightroom antique presets

A couple of years ago I came across Camiel Schoonen’s very well-done set of antique presets for Lightroom/Lightroom 2 (link to flickr discussion page for the presets). They achieve some nice effects, as in my example below from a ride on the Grand Canyon Railway. The modified photo is on top, original beneath:

(link is to flickr photo)

After a laptop transition gone awry, Camiel lost his copies of the presets, but I still have a copy. I’ve forwarded these back to Camiel so he can resume their distribution over at his own site.

Thanks for the great work, Camiel!

Relative sized borders in Lightroom 2

I’m sure the rest of you figured this out ages ago, but it was a new realization to me. In addition to supporting multiple, different borders per image, the LR2/Mogrify border options for Lightroom 2 allow for borders to be applied on three metrics: pixel size, percent of height, and percent of width. That allow, for example, for easy application of letterbox-style top and bottom borders without needing to pay any attention to the cropped dimension of your image.

Screenshot - selecting fixed-percentage borders in Lightroom 2/LR2 Mogrity

The plugin default is in pixels; just switch to percent and you’re off. Something in the range of 10-12% of height seems to suit my eye pretty well.

Lightroom archiving and backup revisited

A while back I described my Lightroom 2 backup routine. A couple of recent blog posts around the web have given me some new things to think about with regard to backup and workflow.

Thomas Hawk writes about his workflow and Eric Scouten follows up with some thoughts. A core part of Thomas’s workflow is quick development of any and all keeper photos from a shoot and immediate export of those images to JPG. He re-synchronizes those JPGs with his Lightroom library and moves all the raw images offline (to a drobo; Jon at blurbomat extolls the drobo’s virtues, too) , and doesn’t apply keywords and such except to the finished JPGs.

Reacting to this, Eric notes a couple of things that get my attention (his emphasis):

  • The act of choosing what’s in and what’s out becomes an affirmative process rather than a process that’s about rejecting photos. Why not make the selection process a happy one?
  • This means the “selects” catalog is always in tip-top shape whenever I need to show someone my current work. Right now, the “main” catalog always contains some number of rough photos that haven’t been filtered out to the archive catalog.

By quickly moving from raw to selected and and “finished” JPG, the process builds the catalog of preferred images and gets the unused images out of the way. And applying keywords and other metadata only to the keepers dramatically reduces the overall metadata workload.

Where does this point me? I have a couple of thoughts:

  • Moving raw images offline, leaving only the JPGs readily-accessible, is a great way to impose some discipline over the “ooh, I might use that one someday” tendency. On the other hand, I’m not shooting in volume like Thomas is; while storage is an issue, there’s definitely a creative trade-off here. Until I shift my raw images to my network drive (usually about monthly), I’ll revisit a folder half a dozen times or more to see if there’s something interesting there that I didn’t see before. Thomas’s workflow, and that considered by Eric, are very different from mine though there is something appealing in it.
  • What this really opens up for me is a little bit of thinking about the intent and purpose of the photos that I take. Making sense of workflow and eventual archiving is a way to be conscientious of why I take photos. Is it because I’m an archivist? Is it for artistic purposes? Is it because I want to build a portfolio? I suppose I have a streak of all of those in me. I like the shooting itself, and I like the workup in Lightroom as well. So for me, it doesn’t quite make sense to workup photos quickly and move them offline.
  • However, it does make sense to develop a solid “keepers” collection as a sort of middle road; this collection gets updated frequently, but whether to keep those images as raw or JPG isn’t something I’ve quite sorted out.
  • What about those “someday” images? I need to consistently flag images with some potential and return to them regularly to re-evaluate.

Thoughts? How do you manage the possibility of future image development with your goals, limited creative time and storage space?

Quick and easy curved borders in Lightroom 2

The post-crop vignette function in Lightroom 2 has much more flexibility than the comparable tool in Lightroom 1 — it’s made to allow for creative use of vignette rather than simply correcting lens vignetting. And you can put it to use to create quick and easy curved frames without calling on any external tools or plugins like LR2/Mogrify.

Sorels

The above image has a white curved border applied by using the following vignette settings:

Lightroom 2 -- post-crop vignette settings

The trick to the curve is the right combination of midpoint and roundness — adjust to taste. And the hard edge of the vignette comes from setting “feather” to zero. To make a black border? Just slide “amount” all the way to -100.

You can make a preset for both white and black versions: Click the big plus button in the preset pane, and select only the post-crop vignette setting to save. Do one for both black and white borders, and you’re good to go.

Quick and easy!

The year in Lightroom, by the numbers

I started last year to play with pulling data right out of my Lightroom catalog. How fun to combine interests in photography with my need to make data out of things. Last year about this time I posted some 2007 photo stats, and with the release of Lightroom 2 I came up with some keyword network maps of my flickr images.

Over at The Online Photographer, Marc Rochkind did some writing about meta metadata and released a tool for OS X that produces much more summary information than I had previously considered: His tool produces by-lens statistics on cropping and aspect ratio in addition to focal length usage. This generated some thoughtful conversation about composing in the viewfinder versus cropping, and Marc’s work spurred me to think more about my own stats, and so I went back to my own Lightroom 2 catalog with the sqlite browser and R to see if I could reproduce for myself some of the more interesting data that Marc’s tool generated. After some tinkering, I think I have a functional, reusable set of R tools for generalized reporting of Lightroom image data.

Like Marc’s ImageReporter, I can filter by image type, picks, ratings, Quick Collection, camera model (though this matters less for me since I have one P&S and one DSLR) and time period, and I added filtering by color label as well — hey, just for fun, even though I don’t use the color labels (I generally get rating fatigue using anything more than picks.)

So, what do I have? First, a reproduction of the stats I checked out last year: Monthly photos and focal length:

The year in Lightroom

I continue to primarily use my prime lenses, and my picture-taking appears to have notched down dramatically as compared to 2007. This is partly because of work, of course, but also because I’ve become much more selective about what I actually keep in the catalog.

We can break out focal length a bit more. For the two zooms that I use on my K100D, what are the mean focal lengths?

> lensFL [1] 5.8-23.2 mm 15 [3] 85.0 mm f/1.8 85 [5] smc PENTAX-DA 18-55mm F3.5-5.6 AL 31 [7] smc PENTAX-DA 21mm F3.2 AL Limited 21 [9] smc PENTAX-DA 50-200mm F4-5.6 ED 121 [11] smc PENTAX-DA 70mm F2.4 Limited 70 [13] smc PENTAX-FA 35mm F2 AL 35 [15] smc PENTAX-FA 50mm F1.4 50

So that’s kind of interesting, suggesting that I use the 200mm zoom at about the middle of its range. But the mean isn’t necessarily informative. Here’s a plot of focal length for one of those zooms:

Focal lengths plot, DA 50-200mm lens, 2008

So, I use the 50-200mm lens primarily for shots at either extreme of its length, and I already have a 50mm fixed lens that takes better photos than the zoom at that distance. Moreover, breaking out just picks with this lens shows a three-to-one preference for 200mm than for 50mm. I think that means I need a long prime. Ka-ching!

I can also consider crop: How am I doing at composing in-camera? Here’s how often I crop, by lens, as a percentage:

	smc PENTAX-DA 18-55mm F3.5-5.6 AL   9.13 %
	smc PENTAX-DA 21mm F3.2 AL Limited 17.67 %
	smc PENTAX-DA 50-200mm F4-5.6 ED    6.93 %
	smc PENTAX-DA 70mm F2.4 Limited    23.78 %
	smc PENTAX-FA 35mm F2 AL           10.71 %
	smc PENTAX-FA 50mm F1.4            24.67 %

And, when I do crop, how much of the original composition do I keep?

	smc PENTAX-DA 18-55mm F3.5-5.6 AL  78.3 %                            
	smc PENTAX-DA 21mm F3.2 AL Limited 81.8 %                            
	smc PENTAX-DA 50-200mm F4-5.6 ED   81.6 %                            
	smc PENTAX-DA 70mm F2.4 Limited    80.9 %                            
	smc PENTAX-FA 35mm F2 AL           83.4 %                            
	smc PENTAX-FA 50mm F1.4            82.5 %

So, I’m cropping quite a bit. As Marc found in his exploration, these numbers go up when I filter by picks. I was surprised that I crop as much as I do with the DA21mm in particular, since I think of my use of it as being mostly for wide landscapes; but even those often enough are a bit crooked, enough to warrant at least some adjustment of tilt —- and Lightroom calls that adjustment a crop (fairly).

Does cropping mean I do a poor job at composing in-camera? Possibly. I have to admit that knowing I can crop gives me a conscientious freedom when I’m shooting, but these numbers give me something to think about. Maybe careful composition will be something to work on as I go forward.

We can cut all this in a few other ways. I’d like to take a look at my common keywords during a given time period, for example, but that will wait for the follow-up post, I think. This is more than enough nerdery for one January 1st afternoon.

Quick Lightroom 2 Tip: Show untagged images

I haven’t used Lightroom 2’s library filter extensively, but it’s really pretty powerful and I found a good use for it this morning. Having tagged a big batch of new images somewhat helter-skelterly, I found myself wondering if I could view just the images that I had not yet tagged, so that I could take care of them and be all done with this round of keywording. The library filter does this handily:

lightroom 2 metadata browser - setting presets

Open the filter pane (hit \ to activate and deactivate it), then select the “text” controller, and set it to “Keywords” “Are Empty”, and there you go: Lightroom 2 will display just the untagged images in the current view. Very handy, and the display will update as you proceed to tag, slipping newly-tagged images out of view.

I saved this library filter view as a preset using the dropdown controller available in the upper right of the pane, so I can quickly retrieve it any time I like.

Update: I see the following in a good post over at the O’Reilly Lightroom blog:

One of the example smart collections that is installed with Lightroom is Without Keywords. Take a look at the image count for this collection. If it’s anything other than 0 then get in there and add some keywords!

Nice! I haven’t much explored smart collections. That built-in collection is a great find.

Dealing with duplicate keywords in Lightroom 2

I’ve noted recently how slick it is to be able to use nested keywords in Lightroom: It’s a piece of cake to select a set of photos, hit cmd-K, and enter “mount humphries > mountains” to assign the “mount humphries” as a child keyword of “mountains”.

However, as noted by a poster to a thread about keywording over on the Flickr Lightroom group, this creates a potential problem: If the child keyword is already used, Lightroom 2 will end up creating a duplicate keyword; you’ll end up with one “mount humphries” without a parent, and one “mount humphries” keyword with the parent of “mountains.” So, by trying to be hierarchical with your keywording, you’ve actually splintered your keywords. Not helpful!

Having already found a fun way to explore relationships between and frequencies of my keywords, it occurred to me that I might have some ready-made tools to help with this situation: The need to find and deal with duplicate keywords.

As this article became more popular, I worked through a couple of alternative methods and organized things a bit further. To date, I describe three methods of identifying duplicate keywords:

  • Full auto: Requires some scripting but is the most expedient way to go about it (and my favorite).
  • Semi-auto: Requires the awk tool to identify duplicates but doesn’t rely on any sqlite3 code to pull from the LR database.
  • Full manual: Uses LR’s built-in export tool and MS Excel to get what you want. Lots of steps, but it works.

After finding your preferred method, read on to see what to do with all the duplicates you identify.

The full auto/scripted method

Thanks to Dieter for putting me on to a much quicker, straightforward way to identify duplicates using SQLite only — no awk:

select count(name) as num, name from AgLibraryKeyword group by name having num > 1

I’ve kept the original SQLite+awk method below for posterity:

First I needed to find any duplicates created — potentially — by the ad-hoc nesting of keywords. A quick modification to my keyword frequency script produces a list of dupes:

# display a list of duplicate keywords in Lightroom 2 cp ~/Pictures/Lightroom/Lightroom\ 2\ Catalog.lrcat ~/lightroom.lrdb /usr/local/bin/sqlite3 -csv ~/lightroom.lrdb 'select ImageCountCache, name from AgLibraryKeyword;' > /Users/alan/lr-keywords.csv awk -F , '{print $2}' lr-keywords.csv | sort -n | uniq -d rm ~/lightroom.lrdb rm ~/lr-keywords.csv

As with the frequencies, this short script makes a backup copy of my Lightroom 2 database, then calls on sqlite3 to extract the list of keywords; the difference is in the 4th line, where I use awk to pull out the tags and then pipe them through two built-in unix functions to print a list of any duplicates. In my case, it yielded the following:

Running the duplicate keyword script

Perfect! A list of keywords that appear to be duplicates.

Note: The script above works with OS X Tiger and requires an upgrade to the default version sqlite. It ought to work with Leopard as long as sqlite is present. Windows? Don’t know; either via cygwin or separate binaries, awk and uniq should be available for Windows, and there is a sqlite for Windows download at the above link.

Skipping the sqlite step

Update 17/Dec/2008: If you’re not eager to delve into sqlite, you can still make this work, but you’ll still need to have the awk tool. OS X users, you should be good to go, since awk comes with the OS; Windows users, you can download awk for Windows. First, manually export your keywords list from your catalog: Metadata > Export Keywords…, and save the file as lrkeywords.txt, and then run the following one-liner script from a shell/terminal:

awk 'BEGIN {FS = " "}; {$2 = $2; if (match($0, /{/)0) print}’ lrkeywords.txt | sort -n | uniq -d

Just as in the sqlite version, this one-liner parses your keywords file and returns the list of keywords that appear more than once and are not identified as synonyms. You can then reconcile the duplicates as described above. I prefer the single-step version that extracts directly from the database, but hope this is useful to a few folks.

The Full Manual Process

If you’re averse to both the sql steps and to using awk, you can use MS Excel to identify the duplicates. I think it’s far more cumbersome than either of the above processes, but it works. Here goes:

Manually export your keywords from Lightroom: Metadata > Export Keywords ….

Open that export file in Excel: Open as a text delimited file, but uncheck all the delimiters; you don’t want excel to parse along spaces or tabs, since both of those characters appear in the file but not as record separators.

You should see a single column of keywords something like this:

screenshot

We need to slightly clean up that column to remove leading whitespace. We’ll use the CLEAN function to do that: In the column next to your keywords column, enter the formula =clean(a1), and then drag that formula all the way down the keywords column.

screenshot

In col B, you now have a whitespace-trimmed set of keywords, but because of the clean() formula, you can’t manipulate it further. Select that entire column, copy it, and then use paste special to paste the column as values into column C.

screenshot - cleaning up keywords list with paste special

Now we’re ready for the final steps: Sort, flag, and filter. Select column C, then go to Data > Sort, and sort ascending by column C. Now, in column D, drag the following formula from the first row to the last: =IF(C2=C1, "!", "").

screenshot - applying the duplicate flag formula to the sorted list

That just fills the column with a flag if column C has a duplicate. It’s a low-budget search, but it works as long as the list of keywords is sorted. Finally, use Data > Filter > Autofilter, and click-select the ! in column D. You’ll now have a filtered list of duplicates from your original keyword list, which you can resolve as described above. Note that you’ll have a number of keywords surrounded by {} or [] brackets; these are keywords entered as synonyms or categories, and you should ignore them when you are addressing duplicates.

screenshot - the list of filtered duplicate keywords

You can see from my current duplicate list that I’ve been working heavily on food-related keywords as we cruise through the holiday season.

All told, that whole manual process should just take a couple of minutes once you have the steps sorted out. Because the steps are manual, it’s not as easily-repeatable as the automated sqlite+awk approach, but it does work. I hope someone finds it useful!

Dealing with the duplicates

Whatever method you’ve employed, at this point you have a list now — let’s check out if it means what I think it means. Switching back over to Lightroom, I can filter for all photos with the “mount humphries” keyword:

Lightroom listing duplicate keywords

Sure enough, I have 12 images tagged with “mount humphries”, and 11 images with the same tag set as a child of “mountains” (as an aside, I see that I have well over a hundred images with the “mountains” tag that could probably use some more granular tagging).

My first impulse was to try to just drag the non-child “mount humphries” into the “mountains” tag; this works, after all, with other keywords. But in this case, it won’t do the trick, presumably because there is already a “mount humphries” keyword there — Lightroom won’t let me add a same-named child.

To reassign the keyword to the parent, you need to take a few more steps: First, click the right-pointed arrow to the right side of the duplicate, non-child keyword; this will navigate to all images assigned that keyword. Then, in the grid view, simply select all (cmd-A), and then check the child keyword to add it to all of the selected images (the checkbox is to the left of the keyword, and appears when the mouse cursor hovers over the keyword), and you’ll see the count increase accordingly. Next, un-check the duplicate, non-child keyword in the keywords panel. You’ll see its count drop to 0. The order of those two check-uncheck steps is important: If you uncheck the non-child keyword first, you’ll end up with an empty selection and nothing to apply the proper keyword to.

Lightroom displaying duplicate keywords unchecked

There! My “mount humphries” keyword as a child of “mountains” is now assigned to all 23 original images, and I can delete the duplicate, non-child tag.

So, with an approach like this, ad-hoc keyword nesting shouldn’t be feared: We can identify duplicates created by nesting, and, in a matter of seconds, apply the same nesting to any previously-tagged images. And, once you’ve resolved the duplicate, any further assignment of the focal keyword will always assign it, appropriately, as a nested tag. Pretty slick, I do say.


About, the short version

I’m a sociologist-errant. This site is powered by Textpattern, TextDrive and the sociological imagination. For more about me and this site, see the long version.

Copyright and so forth: Commenters own their own posts, and linked or excerpted material is subject to whatever copyright covers the original. Everything else here is mine, rights reserved.

RSS feed