Pics or it didn't happen

Pics or it didn’t happen

It’s May again (I know, I don’t know how that happened either, except that it followed April, and don’t get me started on April). Among other things, May is home to the “IronMan in May” competition at my place of work. The goal is to complete a sum of distances equivalent to an IronMan triathlon in the span of 31 days: 26 miles running, 2.4 miles swimming, and 112 miles biking. It’s a fun challenge and makes for a nice opportunity to mix up my workout routine, especially since I’m really, really not much of a runner.

And of course it means data to keep track of! There’s a paper chart on the wall at work, but who would want to use that when there’s such a glut of data visualization tools making the rounds now? Two years ago I used Joe Gregorio’s sparklines tool to plot my progress as I went:

Well it’s 2010 now (don’t get me started on 2009; that’s when I dislocated my shoulder — again — and then got nasty tendinitis to boot, so I didn’t take on IronMay last year), and this year I’m using Processing to plot my month’s data. Like R is really a programming language oriented to data-oriented programming, Processing is a language oriented around visualization (unlike R, however, which makes it easy to access vectors and data frames, Processing requires one to go back to array manipulation, so it took me some getting used to). Now, having seen the spectacular stuff in the Processing gallery I just hope that I’m not insulting the poor software by using it to make bar graphs (that look an awful lot like the old sparklines! I figured I had to start somewhere as I learned something brand new-to-me). This quick Processing tutorial is a great place to start.

Update May 31 2010: It’s a gorgeous day in Flagstaff and I knocked out the last of my run and bike miles on the urban trail this morning. Iron June, anybody?

Update shortly after the prior update: I wasn’t content with just the bar plot, so I tinkered just enough with streamgraph.js to come up with this (click through for the full-sized version at flickr):

IronMay 2010 - streamgraph

Pretty rad, I think.

Good stuff: Tivoli radio

Although I recently gave a disclaimer about my home stereo system, I do enjoy my music. And for the past couple of years, I haven’t loved the way I listened to it. The aforementioned hawt stereo lives with the TV in a little loft space, and the small speakers there can’t really get any good sound into the kitchen, which of course is where I spend a lot of prime music listening time. The MacBook isn’t a bad speaker, but it doesn’t really have the volume or depth to fill the room.

So recently I went in for a Tivoli Model One.

Model One

It’s way cool: It sits on the kitchen counter and can easily fill the kitchen with sound — and good, good sound at that. [ Note that the Model One is mono; since my kitchen isn’t really conducive to a stereo setup, this isn’t a problem for me. One of these days maybe I’ll figure out how to rig up stereo there, and might spring for the Model Two. ] All the knobs are big hefty-feeling switches, the cabinet is sturdy, the tuner knob is geared. I tell you, it feels a little weird to write that it feels good to operate this radio, but man, it feels good to operate this radio.

With an aux switch and Airport Express hooked up to the input, I stream to it from either MacBook or iMac. And I can control the whole whiz-bang thing from the table with the iPod remote. It’s like living in the future. I highly recommend it.

Tasks + Journal in TextMate

A couple of years ago (whoa now, three and a half years ago?!) I commented on using the TextMate Journal Bundle to keep a snippets log. I still use that same snippets file for some references and a handful of things that don’t quite fit in 1password.

When 2010 rolled around, I started up a new journal file to use for a little bit more writing, and quickly found that I wanted to mix content within that file — specifically, I wanted to add tasks and treat them more functionally than straight markdown list items. Henrik Nyh’s TextMate Tasks bundle handles to-do lists quite well, so I cobbled together a combination of the two bundles. A screenshot of some of these integrated journal+tasks lists is below:

JournalTasks bundle screenshot

Combined, I get the nice markdown-compatible writing of the journal bundle, plus its keywords, search, and folding capability, along with the nested list feature of Tasks.

Use the journal bundle’s dts and dtsp tab triggers to start new entries complete with date stamps (and clipboard contents in the case of dtsp). You can write text into any entry, and intersperse lists at any point by starting a line with - (indent with one or more spaces to create sub-lists; Tasks will love it). Toggle list items to complete with cmd-D, and cmd-E will remove expired items.

I like just being able to put lists into markdown-enabled writing, but there’s a nice Getting Things Done application here, as well: Since the folding in the journal bundle is so easy, you can make @context entries, fill them with actions, and fold them to get out-of-scope contexts out of the way. Drop the name of the scope into a keyword, and the Find in Bundle command will produce a nice list of the first entry in a given context that matches a search. It’s simple but workable for GTD, if you’re into that sort of thing.

A variation that’s even more direct GTD-style tasking is to use @context tags anywhere in your entries and then employing the “find tasks” command to display only the desired matches. Use either or both find commands to fit your style and workflow!

Note that the Tasks bundle is inspired by the very good TaskPaper app. If you’re interesting in something a bit more full-on for lists (and GTD) that also has a bunch of extensions and related projects, do check it out. (Since TaskPaper works with text, you can even load files from Tasks+Journal or Tasks right into it and maintain some (but not all) functionality. I’ve done a little bit of further tweaking to maintain that compatibility in my bundle, like change Henrik’s check-mark back to the “@done” syntax that TaskPaper users (and update the bundle’s commands accordingly), which lets me switch back and forth pretty smoothly.

The JournalTasks bundle is available at github.

Pentax K-7 on sale at Amazon

The Pentax K-7, Pentax’s current flagship DSLR, is on big sale today at Amazon:

In-body shake reduction (full shake reduction on ancient lenses!), 720p movie mode, 5 frames per second, and dust, moisture and weather resistant case. This is the first time I’ve seen the camera for under about $1200, so it’s significant savings. Both body-only and kit models are available. I just might pick one up. (And so might you.)

Organizing the digital mega-library

A few weeks back I wrote a bit about our new supra-digital entertainment nexus station, also known as The Blu-Raya Playa. After some more time basking in its HDMI greatness, I have a few more reflections that seem, in natural blog law, to require public consideration.

First, a disclaimer: The audio portion of our “home theater,” such as it is, consists of an eight year-old Sherwood receiver that I still maintain is an extraordinarily good buy for 2002, what with its DTS and coax and optical audio inputs and 5.1 downmixing and full LRF support. It’s connected to the bookshelf speakers that I bought in 1994 to take to college, so when I say that the audio from the Serenity blu-ray is totally sweet, you should take my word for it, because I’m not one of those nutjobs who pays $800 for neutrally-balanced balsawood volume knobs.

Right, with that out of the way, the Sony N460 remains a very satisfying little piece of equipment, but I do have a few thoughts about the whole “online digital library” thing.

First, a better remote would be nice. This one’s a little lightweight in heft. Also, for a device that can stream internet music services, a RF remote would be slick — no line-of-sight requirement to make selections. In the dare-to-dream category would be a smart remote with a display to control services like or pandora from the kitchen. Oooh, iPhone app. Ooooh, iPad app, yes.

Second, every video service is walled off from the rest. Online video is organized by service, not content or category or keyword. I want to aggregate it all up into folders or buckets of some kind. I’d settle for getting this for Netflix only (though being able to roll-up rentals or purchases from Amazon on Demand, too, would be way cool). The current display is just a tile of little tiny cover displays — scrolling through brings up a larger display of title. How excellent it would be to be able to organize all those little pictures of DVD covers, into TV and sci-fi and “serious melodrama” and “re-runs of The Shield.”

Here’s why categorization matters so much: I find I’m really making a library out of the Netflix instant queue, dropping into it movies I’ve been before, movies I’ve heard about, movies I might watch someday, and even documentaries. Previously, the Netflix Instant queue was a list of things tagged, essentially, “I want to watch this.” But having all that material in a device permanently connected to the TV changes that dramatically. It’s not a queue so much as a “I love this or I think it’s cool or I remember watching it in college or I might watch this one day” list. And therefore it’s not a list; it’s a library, and that library needs organizing. And all that stuff? It needs organizing.

The digital mega-library needs organizing, one way or another. Netflix could send category information to these devices, for example, or could allow users to attach keywords; devices could do anything with this information: display tag clouds or let users navigate the categories of video they’ve dropped into the queue, or build sparklines of keyword frequencies or … well, you get it. I think Netflix must ultimately be headed for this kind of system, but its usability will depend on device-level implementation.

[ update: Boxee may present a partial solution here, as the new version claims to be able to better organize material across services. Not having tried it out for quite a while, I downloaded the current release and gave it a go. I do have to say that it’s really slick, and must faster than the prior version I had used for a while. And there is some hint of cross-service integration: Searching for Chuck for example reveals streams from and from Hulu; but searching for Doctor Who, which is represented by half a dozen entries in my instant Netflix queue, turns up nothing, so the integration is at best still only partial. But Boxee does have the most complete overall set of services, so the upcoming Boxee box is most definitely worth watching as an alternative to dedicating a PC to the TV. ]

And the day I can get to all the stuff I’ve queued, rented, and/or bought — via keyword, or actor, or ranking, or genre — without having to back out three menus and find another service? That will be Unification Day, and it will rock. Apple TV, I’m looking at you, though if Sony could do this with some sweet firmware updates to the box we already own, well I’d seriously consider a Vaio. Cross my heart.

This changes everything

Just after Christmas, our DirecTV receiver unceremoniously gave up the ghost. I had been looking into alternatives, and wasn’t encouraged by the lack of incentives for us to stay with DirecTV: In order to upgrade to a DVR, we’d have to pay $200 up front for the new unit (and still wouldn’t own it, as I understand), plus the additional monthly fee. By comparison, a comparable package with Dish Network was substantially less expensive, and they even offer an option to opt out of the 24-month contract — which is still cheaper than our existing DirecTV package.

But that’s not actually my point. My point is that when the first receiver broke, DirecTV promised to send us another one right away, but it would still take several days. And, quite honestly, we were bored now. So we gave ourselves a late Christmas gift: The Sony N460 Network Blu-Ray player (amazon link). Very briefly: it changes everything.

Devices like the Roku have been offered this capability for a while, of course [expanding from initially streaming Netflix, to Amazon on Demand and music via Pandora], and we’ve connected the MacBook video to the TV once in a while for Hulu or a Netflix stream. And more recently there’s a whole crop of blu-ray players that are internet-enabled, with varying service connectivity. Actually having the networked device connected directly to the TV is new to us, and the difference is stunning. I underestimated by a massive degree just how cool and convenient it is to push a button and have a library of streaming content available without screwing around with video cables. The N460 supports Netflix, Amazon Video on Demand, Youtube, and a small grundle of additional services, in addition to NPR streaming radio and slacker radio.

Netflix streaming — arguably the headline app — is excellent. Items in your instant queue are displayed in tiled icons on the screen, and it’s a serviceable presentation, though I’d prefer the coverflow-style presentation that Roku uses; the tiles are a little hard to read, and the lack of ability to organize a very large queue might eventually become a problem. Once a video is selected, the streaming starts up quickly and you just go to town (we connected the N460 directly to our Airport Extreme via ethernet cable). Browsing YouTube also works very well (and looks surprisingly good in fullscreen, too), and Amazon on Demand is simply sweet: We rented Star Trek for $3 and got Hi-Def streaming with digital sound. As Doctor Egon Spengler best put it, yes, have some. All of those singularly are super-cool, but as a package it means a new digital on-demand library. I’m still getting a handle on just how big that is for home video.

It’s big.

Oh, yeah, it plays blu-ray discs, too. That’s cool.

Another year of photo data!

Following in the modest two-year tradition I’ve established (see 2007 and 2008 posts), here is my 2009 photo data from my Lightroom catalog!

[ quick howto: Lightroom 1 & 2 (and 3) databases are in sqlite3 format, which means that freely-available tools can extract data from them. I use sqlite3, some shell scripting, and R (and occasionally excel) to produce summaries of that data. Why? Data offers some insight into the kinds of photos I take. Mostly, though, it’s fun. I’d be happy to expand on the actual code that goes into these plots, if there’s interest. ]

Below is a set of plots that summarize some of this year’s data. Click through to flickr to see the larger version.

2009 photo data!

What’s interesting this year? Well, crop ratios looked pretty similar to last year, so this year, for the first time (suggested in a post by Keitha, whose photos I admire tremendously, and whose Pentax lens set I envy with the fire of a million anti-glare-coated nine-aperture-bladed all-metal suns) I pulled out some information about aperture for each of the prime lenses that I shoot with. You can see these four frequency plots (for each of the Pentax DA 70mm F2.4 ltd, FA 50mm F1.4, FA 35mm F2.0 and DA 21mm F3.2 ltd lenses) in the left hand column of the image. Right off the bat you can see that I shot a lot with the FA 35mm this year (which is confirmed by the “overall lens use” plot on the right column). In fact, I took that lens along as my sole lens on a few long weekend trips to Ventura, CA, and the San Juan Islands, and really loved its performance. It does great at large apertures, but I also used it a lot for street shooting at f/8 and smaller apertures.

Runner-up in frequency this year is the FA 50mm F/1.4, which ordinarily I would say is my favorite lens (and it very much still is; it just wasn’t as convenient a focal length to take as my only lens on those vacations). Its sweet spot [where it’s sharpest but still has a nice narrow depth of field] is about F/4, which is where I primarily use it.

Neither the DA 70mm F/2.4 or the DA 21mm F/3.2 got as much use this year, but I really love some of the photos I took with those lenses. In fact, I carried these two lenses specifically for their light weight and trim size on the Flagstaff photowalk I organized in July.

Car / Cat Ranch house / wide Crow Pomegranite Backside Doorman

How did 2009 stack up to 2008? In terms of absolute frequency, nearly identical! I kept 1308 frames last year, compared to keeping 1340 in 2008. Far fewer of those are picks, or posted to flickr — though a good number are waiting for me to come back to, to finish workup or to make a print.

And that’s it for the 2009 photo stats! I did re-work my keyword network code, so perhaps can follow up this post with a little more about keyword relationships.

If you’d like to know more about extracting and summarizing info from your own Lightroom catalog, please let me know (and check out my other lightroom-related posts)

And, as last year, I hope soon to follow up with a report on my 2009 photo goals, and to set a few for 2010.

Lightroom: Build thumbnails and full-size exports simultaneously

It would be great if Lightroom could string export presets together, to perform operations sequentially or simply to perform several processes on one image to produce several different outputs.

The first use I’d put to this functionality would be generating thumbnails of exported images right along-side the full-sized image — a common task when producing images to upload to a blog, for example.

Absent that kind of flexible functionality, here’s a nice solution for generating thumbnails of images and full-size exports at the same time. Use Jeffrey Friedl’s Run Any Command plugin in conjunction with mogrify: Just insert “run any command” to an existing export preset, and then add the following as the Command to Run (assumes you have LR2/Mogrify installed, which of course you do):

cp '{FILE}' '{NAME}'-small.JPG ~/Library/Application\ Support/Adobe/Lightroom/Plugins/LR2Mogrify.lrplugin/LRMogrify.extras/mogrify -resize 50% '{NAME}'-thumbnail.JPG

The command makes a copy of your original image (which is available to “run any command” plugin via the {FILE} token), with “-small” appended to the filename, and then uses mogrify to resize the copy to 50%. You can optionally adjust the percent to scale or add a “-quality” option to the mogrify command.

You could perform this command more cleanly, without the copy step, if you have ImageMagick installed independently of LR2/Mogrify: The full IM package includes the “convert” command, which allows for output to a new filename. Since I didn’t want to separately install ImageMagick, I just make a copy of the file and then invoke mogrify via the path to the plugin.

Presto! Instant thumbnails, appropriately named, to accompany your export. Pretty good stuff, made possible by the great work of Jeffrey Friedl. Check out all his lightroom goodies.

Today's Lightroom quick tip: Targeted adjustment shortcuts

It’s nice to keep learning things about Lightroom. It tends to have solutions for problems long before I realize that I need them, and so I was happy to find this morning that it has handy shortcuts for all the targeted adjustment brushes.

  • shift-alt-cmd-N: None (cancel adjustment)
  • shift-alt-cmd-T: Tone adjustment
  • shift-alt-cmd-H: Hue adjustment
  • shift-alt-cmd-S: Saturation adjustment
  • shift-alt-cmd-L: Luminance adjustment
  • shift-alt-cmd-G: Grayscale mix

Activating any shortcut will jump your develop panel to the appropriate controller, so you can keep your cursor focused on the image and still see the effects of your adjustments in the sliders or levels. It’s a nice, efficient way to minimize extra mouse movement and make quick modifications.


Check out the rest of my Lightroom posts for much, much more, including more database tinkering, keywording, and workflow.


Energy visibility

Our electric utility recently replaced our meter box with a digital one that somewhat magically sends its data back to the mothership. The cool bonus of this is that we can track our energy usage along a number of metrics. The online monitoring application isn’t super-sophisticated and I’d like more flexibility in how it aggregates data, but it does give us a window into how we use power. Not surprisingly, our usage increases in the morning, evening, and weekends. But what we hadn’t expected to see was just how much the usage spiked when the hot tub timer switched on. Turns out it’s a whole lot easier to not be “hot tub people” when you can see just how much power it’s using (and money costing).

What's in your Lightroom 2 catalog?

A question recently came up over in the Flickr Lightroom group about generating a list of all filenames or selected filenames in a catalog. There’s a pretty nice straightforward bit of SQL that will produce just such a list.

My new favorite tool for poking around in the Lightroom database is the SQLite Manager extension for Firefox. It plugs seamlessly into Firefox and is a very good tool—faster than the old sqlite3 browser, and with the added bonus of not requiring Rosetta to be installed under Snow Leopard. Plug the following code into the “Execute SQL” box (or kick it old school via the command line sqlite3 tool):

SELECT pathFromRoot, baseName, extension FROM AgLibraryFile JOIN AgLibraryFolder ON AgLibraryFile.folder = AgLibraryFolder.id_local

You can do even better by using the sqlite3 concatenate operate in the select statement:

SELECT pathFromRoot || baseName || '.' || extension FROM AgLibraryFile JOIN AgLibraryFolder ON AgLibraryFile.folder = AgLibraryFolder.id_local

That will produce a nicely-formatted list of files:


Still building, we can add in the absolutePath field to get a complete path — very useful for when you have files stored in multiple locations, such as offline storage or backup locations.

SELECT absolutePath || pathFromRoot || baseName || '.' || extension FROM AgLibraryFile JOIN AgLibraryFolder ON AgLibraryFile.folder = AgLibraryFolder.id_local JOIN AgLibraryRootFolder ON AgLibraryFolder.rootFolder = AgLibraryRootFolder.id_local

And, finally, because how often do you really need to see the paths for all of the files in your library, really?, we can limit the query to just files in a quick collection. This takes a couple of extra steps, because we can’t directly get to the keyword data from the path data:

SELECT absolutePath || pathFromRoot || baseName || '.' || extension FROM AgLibraryFile JOIN AgLibraryFolder ON AgLibraryFile.folder = AgLibraryFolder.id_local JOIN AgLibraryRootFolder ON AgLibraryFolder.rootFolder = AgLibraryRootFolder.id_local JOIN Adobe_images ON AgLibraryFile.id_local = Adobe_images.rootFile JOIN AgLibraryTagImage ON AgLibraryTagImage.image = Adobe_images.id_local WHERE tagKind like "AgQuickCollectionTagKind"

And there you have it. One more fun trick you can pull thanks to the sqlite3 database that underlies your Lightroom 2 catalog.


Check out the rest of my Lightroom posts for much, much more, including more database tinkering, keywording, and workflow.

Replacing Edit in TextMate with QuickCursor

After upgrading to Snow Leopard on my MacBook, I found that the Edit in TextMate hook wasn’t working. There were a few threads about re-enabling it, but nothing definitive, and the support page suggests either running any target applications (those in which you want to use Edit in TextMate) in 32-bit mode, or perhaps trying some uninstall-reinstall voodoo. Since neither of these were ideal, I thought I’d try the “last-resort” suggestion, QuickCursor.

After a few minutes of use, QuickCursor seems to me to be a great improvement over Edit in TextMate — far from an “if everything fails” option, I prefer it for most of my uses.

  1. Installation is easy. Download, copy to Applications folder, and run it.
  2. Configure: I set it to load on bootup, and it automatically found TextMate as one of my editor options. I assigned TextMate the same keyboard shortcut that Edit in TextMate once occupied (cmd-ctrl-E).
  3. Use! From a Safari field, hit the shortcut and up pops a TextMate window; edit away, save, and your text appears in the Safari field. So far, just like Edit in TextMate, with the added bonus of being uncomplicated and functional in Snow Leopard. But here’s where it improves on the original: Edit in TextMate required the “target” window to be in focus in the target application; that is, when using Safari with multiple tabs open, the tab with the target “edit in” field had to be the active tab. This meant that if you opened an “edit in” TextMate window, then flipped through a few tabs to find something, you would have to relocate the target tab before being able to save from the TextMate window. QuickCursor doesn’t have this limitation: You can open multiple editing windows from multiple tabs, edit any/all of them, and save your edits without worrying about which application or tab is in the foreground. Bingo!

There are a couple of important caveats to QuickCursor that may make it not an ideal solution for some users (my bolds):

QuickCursor depends on two technolgies. For reading/writing data from the original application is uses the accessibility api. The nice thing about that API is that it’s not a hack, it’s a supported API. But unfortunatly not all views support the accessibility api (at least not read/write of the text content). And in particular webkit views don’t support it. And that means that tools that use webkit as their editor (such as won’t work with QuickCursor.

That means that Firefox doesn’t work with QuickCursor, either, since it doesn’t use the accessibility API. Since I’m a Safari user, and never much used Edit in TextMate for Mail, QuickCursor is pretty spot-on ideal for me.

Always more to learn about Lightroom

I’ve been reading the comments thread of this great post at Scott Kelby’s Photoshop Insider blog. It’s absolutely chock full of concise tips for getting more out of Lightroom, and it drives home to me that there’s always lots, lots more to learn.

Here’s my favorite new-to-me tip (Thanks, Scott!): Did you know Lightroom 2 has a ‘solo mode’? It’s a really fantastic way to eliminate scrolling up and down through the set of side panels. Ctrl-click beside a panel’s name and select ‘solo mode’ to turn it on, and then only the currently-selected panel will be expanded. Select any other panel’s name to auto-expand it while collapsing the prior panel. In conjunction with using cmd-1 through cmd-7 to toggle panels, it’s (to me) a nearly unbeatable way to navigate the panels with minimal mousing. (Note this works in both Library and Develop modes.)

MacHeist 3

The MacHeist 3 bundle is only available through April 7th, so if you’ve thought about it and haven’t made up your mind — or are just now hearing about it for the first time — well it’s time to decide.

This year’s MacHeist currently includes, for the low-low-edge-of-your-seat price of $39, OS X applications like Acorn (lightweight complement to PhotoShop, scriptable with Python, cool) and World of Goo. Total bundle value according to MacHeist is over $600.

So amble on over and check it out.

Facebook network visualization

Quick ‘n dirty visualization of the clusters of relationships among my facebook friends:

Facebook network visualization

Data generated with Bernie Hogan’s My Online Social Network app on facebook, and visualized with GUESS. Good stuff, Bernie!

Thanks to Marc Smith — he’s one of the nodes up there — for the link to the flickr version of this image over at Connected Action.

Brayden King and Kieran Healy (they’re up there in my visualization, too) have posted their own plots over at orgtheory: one, two.

About, the short version

I’m a sociologist-errant. This site is powered by Textpattern, TextDrive and the sociological imagination. For more about me and this site, see the long version.

Copyright and so forth: Commenters own their own posts, and linked or excerpted material is subject to whatever copyright covers the original. Everything else here is mine, rights reserved.

RSS feed