Misdirection For Using GPS Tracklogs And Lightroom’s Time Zone Offset Feature…

GPS: 46°2’42” N 9°21’23” E

Looking Into Switzerland

Photographs © George A. Jardine

Getting comfortable using GPS when I travel and shoot has been creeping up on me for several years now. It is a slightly peculiar beast, and putting your finger on the exact purpose GPS serves in photography can be a bit tricky. Combine that with the somewhat obscure nature of how timestamps work, and you’ve got a subject you can sink your teeth into. (And just as quickly, you can get it wrong. Which unfortunately, is what I found today in a tutorial on the esteemed AdobeTV.)

Ever since the early versions of Lightroom, we’ve had an innovative feature that linked GPS metadata to Google Maps, and it was that feature that first prompted me to purchase a GPS unit and start working with it. But you still had to encode your images with GPS metadata before you could do anything with it. At the time, geocoding photos was not nearly as easy as it is today. Looking around for a little help, one of the engineers on the Lightroom team recommended that I try a program from Houdah Software, and that got me moving in the right direction. (HoudahGeo is still my favorite way to pull .gpx tracklogs from my Garmin GPS unit, which surprisingly, Lightroom does not yet do.)

GPS: 39°41’49” N 104°58’9″ W

Early Frost

Early Frost

With Lightroom 4 we now have the Map module, which adds its own special twist to the mix. The Map module lets you drag photos directly onto the map, embed GPS metadata, reverse geocode location information, filter and select photos within any visible map area, and all sorts of other cool tricks. And in general, they’ve made it all pretty easy to use.

But there is one, small fly in the ointment. When I first started sniffing down this path, I was having a bit of trouble figuring out how to use Lightroom’s Time Zone Offset feature. And it goes back to that timestamp thing. The heart of the problem is that a timestamp is just a timestamp, and can only tell you what time a photo was shot, in some local time. Most cameras do not store the time zone that it was in at the moment of exposure. And if they do, it’s stored in a proprietary metadata tag that is not readable by most software. So the timestamp is completely ambiguous.

Pull Quote

From what I understand, this incredibly inelegant situation is not the fault of Adobe software engineers, or of Japanese camera manufacturers, or anyone else you might think to point your finger at. But rather, timestamps are what they are because the EXIF specification simply does not allow for a local time zone entry!

Some cameras do allow you to set a time zone on the menus, but really that’s just a red herring. There is still no context recorded in camera metadata to give the timestamp meaning. (Nothing in EXIF, anyway, that is used by Lightroom. See Manfred’s comment below.)

GPS units take a slightly different tact on the problem. They ignore the crucial idea of context too, but they do that because they can. GPS units simply record all timestamps as if they were in one time zone that never changes: UTC. (UTC = Coordinated Universal Time. Same as GMT or Zulu time, whatever you want to call it.) Sure, it’s true that you can set a time zone on your GPS unit, but that’s just so that you can read a local time on the GPS display that makes sense. This time zone setting has absolutely nothing to do with how the GPS understands where you are, or how it records that position into the tracklog. It simply records everything in UTC. Which makes sense. After all, it is a global system.

GPS: 42°56’45” N 122°10’9″ W

Crater Lake

Crater Lake, OR

Given all of that, the problem should be coming into a bit better focus now. In order to match up the timestamps created by your camera to the timestamps recorded by your GPS unit, your computer needs more information. It needs to know what the timestamps in your photos mean, or… put another way, it needs to have the context that comes from the time zone.

HoudahGeo has a very straight forward way of obtaining that bit of info. The moment you try to load any photos into it for geocoding, it pops up a small dialog and simply asks for it. It also provides a starting point (an assumption), by looking at your computer clock, and pulling the local time zone from there. And it puts the assumption right in front of you for your approval. The text in the dialog says “Camera time zone:”, and there’s a pop-up menu, conveniently set to your local zone. Or whatever time zone your computer is currently set to.

GPS: 36°53’7″ N 104°25’59” W

Decals

Not My Motorcycle!

So it’s unmistakable. HoudahGeo is asking you to verify the time zone of the photos you’re loading, the moment you try to do anything with it. There’s no escaping it. If you’re sitting in your hotel room in China, and you’ve been diligent enough to set your computer clock to the local time zone, (and… you’ve set your camera’s clock to the correct local time, before you started shooting) then geocoding is a slam dunk. You just load the photos, then load up the .gpx tracklog from your GPS device, and HoudahGeo matches them up for you, calculating the offset from the unit’s GMT-based timestamps to your photo’s ambiguous EXIF timestamps.

But then we come to Lightroom. Once you’re in the Map module and you’ve loaded up your tracklog, if you’re still in the same time zone that the photos’ timestamps are in, Lightroom’s Auto-Tag feature will work perfectly for you. My trouble with it is that there’s not a hint anywhere that Lightroom is making the same assumption as HoudahGeo, but that’s exactly what it does.

Danger, Will Robinson

Now, I agree that the software has to start somewhere. But there are two major problems here. First, Lightroom doesn’t give you a clue that it’s making an assumption about the time zone, or that you might need to match these two things up. (Especially for beginners, that’s a problem. HoudahGeo asks you to verify the camera time zone every time you use it.) Second is that, in general, when I’m traveling and shooting, I don’t want to be thinking about GPS metadata and time zones. I want to think about those things once I’m back home.

After doing a bit of thinking on the subject of timestamps, and after polling dozens of my friends and customers who regularly shoot with GPS, I found that I was not the only one. Most photographer’s would rather worry about this stuff when they get home. Which means that Lightroom’s assumption about where to get a useful time zone will nearly always be wrong.

This means that if you want Lightroom to auto-tag your photos to a tracklog, you’re going to have to tell Lightroom where the photos were taken relative to the computer’s current time zone. Thus, the Time Zone Offset feature, which is not exactly self-explanatory. (Also confirmed by my poll.)

Now, I’m not writing this to point fingers at the UI designer or the engineers. There are lots of other aspects of Lightroom that will ensure my job security as an educator. But I’m writing it because just this morning I watched one too many video tutorials that got it so utterly wrong, I couldn’t stop myself.

GPS: 38°48’51” N 115°17’46” W

Desert Love

Desert Love

This feature is not that complicated, and it deserves to be understood because it’s incredibly useful. It’s just been hindered by a really bad user interface, and further obscured by tutorial jockeys who won’t take the time to do a little research. The easy way to think about it is that you have to tell Lightroom how many hours apart the time zone of the photos is (or was), from the time zone your computer clock is currently set to, before auto-tagging will work. And rather than show you a step-by-step here on the blog, I’ve taken 6 minutes out of tutorial #7 from my Catalog Management series, that shows you exactly how to do it.

It’s a free video, and you can watch it by clicking here.

Does Library Folder Organization Matter?

Glass

Photographs © George A. Jardine

I keep reading from all the usual industry pundits that if you’re going to conquer the elusive goal of truly effective library organization, then you will have to adopt this or that folder structure. And although I’ve written a few myself, I’m beginning to rethink my position.

Even if folder organization does matter, there is very little consensus on the issue. Every photographer seems to make up their own library structure as they go along, and the world keeps right on turning. Photographers succeed by following their instincts and finding their own voice, so why would you expect them to behave any differently when it comes to organizing their digital libraries?

The important thing (in the short run…) is that you can find your photos. And so that should be the starting point for a discussion. Think about how you generally try to find any given photo or group of photos, once you have tens of thousands of them, and that might lead you to a surprising answer. (More about the long run later.)

pull quote

This goes back to an idea that I’ve been writing about for some time now; the way that we’ve learned to organize things in the real world tends to shape our thinking about how we should be organizing our digital world. Which is only natural. But does that approach make sense, when it comes to digital photos?

Think for a minute about where that thinking has taken us. In the real world, of course we organize by subject. We have drawers for specific kitchen utensils. And drawers for envelopes and stamps. We have places for our sox, and places for our underwear. And in the physical world, that makes sense.

Japanese Kitchen Knives

When we sit down at a computer, our natural instinct is to begin organizing by subject there, too. And this goes back to the beginning.

When desktop computers came along, a graphical interface with file and folder icons that you could drag around to “organize” was a huge innovation. It gave us a way to organize our computer documents that emulated the way we do things in the real world. And it worked great, at least for a while. It helped “the rest of us” get up to speed with computers. But once you had thousands of files, the system started to break down. The virtual mess inside of our computers became larger than the real mess in our offices.

I don’t know how many people will remember this, but there came a time when both Apple and Microsoft declared that future operating systems would eventually be just one big database. That files and folders would be going away, and when you needed something you would “search” for it.

Despite the fact that this probably was a very good idea, getting rid of the file system never got off the ground. (Until iPads came along, but that’s another story.) By the turn of the century, files and folders were just too ingrained to be taken away from us. Eventually Apple was able to give us a better way to work with certain types of digital media by using a database, and iTunes is probably the best example of that. In the case of your music, iTunes takes care of where it goes, and basically, you don’t care. When you want to find a song, you find it by artist, or album, or in a playlist. Which are all just methods of searching a database.

Random Materials

The logic of that seems like it should be inescapable. Once you have thousands upon thousands of “assets” in the computer, they essentially all look the same. And so good metadata becomes useful—even required. But photographers still tend to think in terms of organizing their digital photos into “places” (folders) by subject. After all, when you walked into a film library, you would know where to find a given piece of film, because you had given it a place. You gave it a place in the library, because in the physical world, you didn’t have any choice! The film has to go somewhere, and if you didn’t organize by subject in some form, you would never be able to find anything.

When it comes to computers, the inherent limitations of that approach should be pretty clear. But somehow that doesn’t prevent most photographers from following the same instinct when building a digital library.

345 Park Ave

Back in 2004 working at Adobe Systems, I was busy interviewing photographers, and thinking about their film libraries. We were in the early stages of Lightroom development, and wanted to understand how photographers thought about photo organization. The approach of one prominent underwater photographer left a lasting impression on me. He had shot 35mm Kodachrome all over the world, covering dozens of subjects. Each slide was filed in poly slide sheets, in stored in filing cabinets, by subject. The determination of that subject was of course, subjective, with no pun intended. If it was a photo of sharks, it was filed with other shark photos. But if the photo was taken in the Caribbean, it might get filed in that folder, by location. (Location is frequently treated as a subject by photographers, further complicating the issue.) If it was shot in the Caribbean and the photo had sharks in it, he had to make a choice. The original might go into the Sharks folder, while a film duplicate of it would also go into the Caribbean folder, and another might go into the Coral folder. Each with hand-written notes on the slide mount indicating where the original was. (Metadata!)

pull quote

One obvious downside to this strategy is that each roll of film was subject to being split up across dozens of filing cabinets, but such were the necessities of cataloging real objects in the real world. (One of the ideas that we get to in a moment, is that preserving the context of shoot order and chronology in digital libraries becomes very easy, and gives you a huge advantage over film libraries that are organized by subject.)

Anyway, back to digital photo libraries. Earlier I said that the folder organization you use doesn’t matter, as long as you can find what you’re looking for. And that truly is the heart of the matter. Because I’ll bet that if you can put your eyes on any given photo that you need in a moment’s notice, it won’t be by digging around in file folders. You will find it by using a search of one form or another. Even if you do sort your photos into folders by subject or location, once you have thousands upon thousands of them, the idea of where those bits are stored becomes meaningless. (Aside from the computer UI trick of file and folder icons, “where” is a meaningless concept in the computer anyway. But I digress.)

Things

So if that’s the case, why do most photographers still sort their digital photos into folders by subject? Nearly every one of my friends, and every photographer I meet, still has top-level folders in their library, separating photos by subject or location very much the same way you would in a film library. Stock photos in this folder. Commercial assignments in this folder. Photos taken in NYC go into this folder. And photos of their family, or their partner, into another folder.

But what happens when you take a photo of your wife or husband while traveling to NYC? Which folder does that go into? And even though nearly every photographer working in digital has figured out how to use keywords to solve this rather than duplicating actual pixels, most digital libraries are still segregated by subject.

pull quote

I believe this is true simply because most photographers haven’t thought through why they do it this way. A set of requirements for your library structure can easily be created by looking at your personal workflow, and asking yourself how you typically set out to find your pictures.

Pumpkins

For me, it’s all about keywords, or collections. If I’m looking for one of my “best” photos, that’s almost always going to mean one that I’ve used in a presentation, or published in a blog posting, or in a web gallery of some sort. And so I can always find my best photos within seconds, by looking in a short list of collections. (I stopped using star-ratings to try and create qualitative rankings a long time ago, once I realized how subjective they were. A 5-star photo in the context of any one day’s shoot, would almost never be a 5-star photo in the context of my larger library.)

Then, if a visual search in my collections doesn’t get me to the exact photo that I’m looking for, it usually leads me to a photo that I know to be from the same group of photos, or the same “shoot”. In this case I right-click on the related photo, choose Go to Folder in Library, and this takes me to the shoot folder. Once I’m there I see all the photos from the shoot, outtakes and all, in the exact order in which they were shot. With this context I can always get to the one photo I’m looking for, in just seconds.

If I am looking for a photo that isn’t in a collection, I’ll search by keyword. When I do a keyword search, it’s going to be a subject like “bridge”, or “wine”, or it’s going to be a location (“Mendocino”, or “Bangkok”), or maybe a person’s name. If a text search returns more than about 100 photos from my library, then chronology becomes important. At this moment the search again becomes visual, and seeing search results in chronological order gives me the context I need to be able to very quickly scan the grid, and zero in on the photo that I’m looking for. Usually within seconds, and even if it’s an outtake.

pull quote

And so in the end, what purpose does segregating actual photo files by subject really serve? (Other than to help you feel that you have everthing in “place”?) Once you’ve built a real library with tens of thousands of photos, do you ever really go into a folder looking for a photo by name? Or do you ever really need to copy or back up bits and pieces of your library by subject, as opposed to shoot folder?

I’m sure some photographers might answer yes to those questions. But for my library requirements, all I need is that my file and folder names give me enough metadata to sort chronologically, both in Lightroom, and in the file system (for archival and legacy purposes). For context in visual search mode, shoots are kept together one folder per shoot. And this is the smallest unit of folder organization in my library. Derivative files (panoramas, composites, etc.) are also managed by Lightroom, right in the same parent shoot folder as the source photo.

Does the fact that you have different file types or project types (such as panoramas or layered composite files) intermingling with the raw files in the shoot folder bother you? I say get over it. When do you ever need to look in there anyway, once you have a more logical way to find the bits that you’re looking for?

Do the pants in the back of your closet that no longer fit you, or the knives in your kitchen that you no longer use cause you to lose sleep at night?

End

A Few Thoughts on Time Stamps, and a New Year’s Resolution…

Two Burmese Women

Photographs © George A. Jardine

Time stamps are hard.

They shouldn’t be, but they are. Nearly every photographer I know has trouble getting their time stamps right when on location. To make matters worse, I never remember to set my camera’s clock for changes in Daylight Savings Time. Then when you get back home, you sit down at the computer, and you’re once again faced with that question. Should I change these? How should I change them? And if I do decide to change them, should I write the changes back into the proprietary raw files?

Uggh.

Over the holidays I took several days to go through my entire digital library from top to bottom, to finally try and clean it up, mostly from the standpoint of folder organization and file names. And it was well worth the effort. But in the course of literally examining each and every digital shoot going back to into 2003, I found that not only did I have lots of folders with incorrect shoot dates and missing metadata, but I also was reminded of how many of my photos had incorrect time stamps. And because I had traveled overseas a fair amount, those incorrect date stamps frequently meant incorrect file names. (See my previous posting on using time stamps in file names.) So I decided it was time to fix that little problem, too.

It proved to be an interesting exercise. And I wouldn’t be writing this blog posting if my conclusion was simply that ‘you should remember to set your camera’s clock’…. or anything as simplistic as that. In the end (again…. more than a week later) I think I finally hit on the solution.

So, here’s a couple of questions for you. If you regularly travel and photograph, do you set your camera’s clock religiously? If you do, do you set it before you get on the plane? Or do you try to set it immediately after you hit the ground, before an interesting picture presents itself? If you set it before you get on the plane, what do you do with a shot like this, that you are lucky enough to get out the terminal window during a layover in Tokyo?

Or, one like this, that you actually took out of the airplane window? (Cliché, I know. But I just couldn’t let this perfectly random arrangement of puffy little clouds get away.)

Let’s see… what time zone was that one in? And at this point, does it really matter?

Or…. let’s say that you’re traveling cross country. When you’re crossing time zones, are you religious about setting the camera’s clock forward or back right when you cross the line? And if you do, does it sort of make you crazy that then you have some pictures that appear to be out of sequence because of the time change? (‘How could this frame, have been taken before this one???’)

The more you dig into it, the crazier it gets.

Don’t even get me stared on Daylight Savings Time. What a strange idea that was. DST drives photographers nuts, me included. Ultimately, you might get to the place where you’re wondering why the camera doesn’t simply take care of all this time stamp stuff, like our laptops and cell phones do. And eventually they might.

But until then, back to my screwed up library. I started sniffing around in there, starting at the top of 2004, with my very first digital camera purchase. A Canon Rebel. By this time I had been growing orchids in California for several years, and I was particularly proud of this one.

The photos of this orchid are the first frames that I have saved from this camera. They are in a folder labeled 20040321, but I honestly have no way to know if they were indeed shot on that particular day in March. I have nothing to correlate them to except the folder name. It’s certainly possible… but again, does it matter? Looking just a little bit farther, I see that the time stamps reveal a different problem. They read 1/1/80, at 12:01 AM.

Hmmmmm. Not likely. I do remember shooting this on my porch in bright afternoon sunlight, and so it was clearly not in the middle of the night, and I would never have set such a time stamp myself. Also, it looks suspiciously as if they were just inheriting some default camera or computer starting date that might be applied to a photo with an unreadable time stamp, or something like that. Looking at the metadata with ExifChanger, and I see that the Exif.DateTimeDigitized and the Exif.DateTimeOriginal fields are both empty. OK…. so that probably explains the default 1/1/80 date. I guess we’re going back into prehistoric time with this camera, when Canon was just shifting over from the .TIF raw format that they used in the first 1Ds. And despite the fact that these .CRW files do not have any EXIF time stamps, they do have an embedded TIFF.DateTime tag. And that’s where Lightroom seems to be picking up the 1/1/80 exposure date. Subsequent shoots with this camera also have TIFF.DateTime tags… and those dates correlate with the dates in my shoot folder names, so that makes me feel better.

Anyway, it was not the early digital captures taken with the Rebel that I was worried about. The weird or missing time stamps was a curiosity, but I was not going to let that consume me. It was a bit later that year that I started doing some serious shooting with an assortment of random cameras that I was able to borrow from the Photoshop team. Of course the Photoshop team had all the cool new cameras, and I could check one out for a day or two, anytime I wanted. And these cameras were a heck of a lot better than my Rebel.

Having a cache of the very latest 1Ds and 1Ds MKII bodies at my disposal was too much to resist and so I tried to get out with these cameras and photograph as much as I could. It was with one of these borrowed cameras that I captured this shot of the Golden Gate Bridge, which I still use as my desktop photo, no matter what color of gray the printing geeks say I should put back there.

I continued to shoot with this somewhat random bunch of cameras for the next 18 months. But again, I never really bothered to look to see what their clocks were set to. As it turns out, some of them were set properly to PST, while others were set to EST, probably because they came from Canon or Nikon reps in NYC, and no one else on the Photoshop team was bothering to set them either. In any case, I was busy shooting with a different camera every week, sometimes in California, sometimes in Denver, and occasionally on the east coast. And so you can imagine the random assortment of time stamps I was collecting.

More determined than ever to straighten out this mess, I started going through my library shoot by shoot looking for clues that might help adjust the time stamps. If I could find just one clue in each shoot, or even just one clue per camera serial number, I would then have a strong enough correlation to correct all the photos that I had taken with that one camera body. I didn’t know this when I started, but two methods eventually revealed themselves as useful in finding such time stamp correlations. First, with the help of timeanddate.com, locating either a sunrise or sunset shot in a group gave me a very strong clue about what the time offset should be for any given shoot. The date stamp on the photo of the Golden Gate Bridge was 3/4/2005, and that matched the date in my folder name. But the time stamp was 9:05 PM, which couldn’t be right.

This page on timeanddate.com showed that sunset on that day in San Francisco occurred at precisely 6:07 PM, and here I was shooting right into the setting sun. So this correlation told me that this particular camera had been set to EST at the time of this capture. And then being able to identify all the shoots that I had done with that camera by its serial number (which is recorded and searchable in the EXIF metadata…) gave me an easy way to correct several other shoots.

But the more surprising clues that I started finding here and there were wrist watches and wall clocks that I had inadvertently captured.

Not a photographic work of art, but in the process of trying to record the wines at this particular event, I ended up with a peek at my watch, that I could then correlate to the time stamp… years later.

In October of 2005 I finally bought a Canon 5D and began shooting with that. Did I ever bother to look at the camera’s clock? Not a chance. And I shot with it for nearly a year, in time zones from California to Iceland without ever setting it. In reviewing my shoots from Iceland, catching a glimpse of a wrist watch for correlation was a lot easier than trying to determine the precise moment of sunrise or sunset. With nearly 18 hour days, sunset was not so much a precise moment, as a long, slow descent into semi-darkness with the sun never truly seeming to set. In this photograph, I found correlation in two wrist watches, and zooming in to 1:1 clearly showed the correct local time to be 9:36 PM.

I admit that it wasn’t until I started capturing GPS information in 2008 that I finally began to pay attention to what my camera’s clock was set to. At this point I was forced to, in order to help the GPS software make the correlation of GPS coordinates to the correct photos. And if you were moving, that time stamp correlation had to be down to the second, or the coordinates you were matching up to wouldn’t be very precise. This is when I started photographing the display of my GPS every once in a while, so that I could know the exact offset of my camera clock relative to GPS time.

And in the long run, this is what finally brought me around to my solution. You see, when you dig into it a little bit, you realize that there is nothing magical at all about the camera’s date and time stamp. The camera’s software expects you to set a local time, and there is absolutely no correlation to GMT, or offset for DST, or any other invention of mankind. It’s just… a local time stamp, that may or may not be “correct”. And nothing more. The GPS unit, on the other hand, is always simply recording GMT. It only requires a time zone setting so that it can show you… the local time. Open up the .gpx file in text editor, and it’s all recorded in GMT. As far as the GPS unit is concerned… there really is only one, true time. And that’s GMT… or UTC (Coordinated Universal Time) as most people in high-tech like to call it.

There really is… only one standard time, no matter where in the world you are. Everything else is derived from GMT. That… makes sense!

Long story short, it took me about a week to correct my entire library. And in that process I found myself asking, over and over again, why should I ever have to struggle with this… on the camera? I never seem to get it right anyway, and just when I do, Daylight Savings Time rolls around and screws it up again! So I wrote a few of my friends who travel and shoot a lot, and asked: Do you always try to set your cameras for every trip? And, how often do you forget? And, does trying to keep on top of it drive you crazy? And the answers were all pretty much the same. To the first question, I got about 50/50. Some do, some don’t. To the second question, about the same. 50/50. And does it drive everyone crazy? Yes. That part was nearly universal.

The fact that the GPS is always, only recording one standardized time caused me to wonder why I just didn’t do the same. Why not just set my cameras to GMT, and then forget about it? It’s when I get home and I’m in front of the computer that I want to worry about what the time stamps are, anyway. Why not just standardize everything to GMT and then fix it later?

And so, that’s my solution. My New Years resolution is to synchronize all my cameras to GMT, and never look back. From now on, I’ll be adjusting my time stamps to the place and time of the shoot, after I get home. And I’m pretty sure I won’t forget to do that, because my file naming depends upon it!