Spaced Out :: 12 2015

Following on from last week’s blog, the first stage in putting together the timelapse video with inset map is to geocode the photos. This post entirely uses the quite remarkable ExifTool for manipulating photo EXIF data that’s embedded within images. It’s fast, has extensive capabilities and is very well supported. it’s command-line only although there are numerous front-end GUIs for it, including both Geosetter (which I’ve mentioned before) and ExifToolGUI. That said, for processing the large number of images for a timelapse it’s much easier to use the command line.

Pre-Requisites
GPS tracklog
photos

1. Timeshift

Step 1 is to timeshift your images. All location information is stored in the GPS tracklog, however because GPS uses the extremely accurate clocks on board the individual satellites to calculate position, the log also stores time. They are usually supplied as GPX (GPS Exchange Format) files which are standardised XML files for GPS data transfer. Most GPSs natively support this now, but if you have an older device you may need to convert to GPX. GPSBabel should be able to handle this for you.

This stage is critical in order to synchronise the time tagged in the EXIF data of the photos to the GPS. The GPX files often record speed as well, so I scan through the GPX file (its text) looking at the start of the data when speed goes above 0 mph. That’s the start of the ride and, within a few seconds, your first photo. Write that down! Now load the first photo and look at the EXIF data (I use Xnview) “Date Modified” tag (in fact several dates can be recorded in the EXIF data, but we’ll be setting them all the same) and write it down. Now check whether those two dates are the same - if they’re not then you need to timeshift your photos. My GoPro had reset itself to the default firmware date of 1 January 2009.

You now need to calculate the time between two dates. I’ve used this website although Excel can do it as well. My images were 866 days, 18 hours, 26 minutes and 25 seconds out!!! The ExifTool command to timeshift this is

exiftool -AllDates+=”0:0:866 18:26:25” .

(the period tells it to process all files in the current directory). If you are doing a multi-day event then all your photos may or may not be consistently out. It turned out the camera reset its clock again halfway through so I then needed to use a different timeshift value. And, then, on two further days there were sections were the clock was wrong. Not sure why but it needed fixing.

2. Geocode

With the photos now correctly timestamped you can geocode the images. ExifTool is pretty clever here in that it reads the time, scans the entire GPX file finding the nearest times before and after the photo and then linearly interpolates between the two GPS locations based upon the time offset. It then writes this information back in to the EXIF data. It also has the option (geosync) to cater for timezones (as GPS uses UTC). The command here is:

exiftool -geosync=+01:00:00 -geotag ..\18_raw.gpx .

3. CSV Output

With all the photos geocoded the final stage, ready for creating the inset maps, is to produce a file listing with all photo coordinates. ExifTool can again do this:

exiftool -csv -n -filename -imagesize -gpslatitude -gpslongitude ./ > coords.csv

Later this week I’ll cover creating the inset maps and then combining all the elements together to produce the final video.

Last month I blogged about an update to my workflow for creating a timelapse and particularly adding overlays (both graphics and text from EXIF data) and solving problems with dates being out-of-sync by timeshifting them.

Well that flurry of activity made me think about the data I collected for my Durness-to_Dover bike ride. For that I used a GoPro attached to the front of my bike and took a photo every 30s along the entire ~800 mile route. I turned those in to straight timelapse videos and thought nothing more of it until recently. In tandem with the photography my biking partner had a cycle satnav with our route planned on it that also kept a tracklog of the route we actually took. What struck me was that the GPS tracklog would have had the correct times stored in it and that I now knew how to timeshift the images. A further look at ExifTool showed that it could not only write coordinates in to the EXIF headers of the photos, but also read the tracklogs straight out of the GPS, find the nearest points based upon time and interpolate between them to give a very good estimate of actual position.

Critically then I was now able to use the tracklog to timesync the photos and geocorrect them. What I thought would could now do was:

1. Time Overlay: insert the time of the photo, having that increment through the video
2. Inset Map: create an inset map with a point showing the position of the photo. This would then animate over the video, moving as day progressed.

Over the next few posts I’ll outline how I built this up (with a little help!), but below is the end product, so you can see what the final result was!

Wanted to point people in the direction of a friend of mine back from the dim distant days of my PhD in Sheffield - Paulo was on a funded PhD from Brazil and I distinctly remember working long hours in the remote sensing lab, with “After 8” breaks in the afternoon.

Anyway, enough nostalgia!! Paulo works as an engineering geologist in the government, but has reduced the number of hours to allow him to work more as a musician. The soundcloud embedded player below links to his latest EP, URBANA IDADE. If you like Satana, then this is very much in that style. I particularly like tracks 1 and 4, with some cracking guitar riffs. Great for an evening of mellow chill.

Three years ago I blogged about using FAT32 beyond the 32Gb limit to format large external drives. The rationale for this was to have an disk file system that worked across operating systems and didn’t run in to the complete drag that is NTFS file permissions.

Well step forward three years and my 500Gb drive is starting to be (very) full and needed upping to 1Tb. That and my increasing need for security meant going in search of something more secure for my external drive - the iStorage diskAshur fitted the bill perfectly with a keypad locked drive, real-time Hardware encryption (256-bit AES) and some shock protection. Great bit of kit and only a little pricier. But…. what file system to have??

Well the reason for moving on from FAT32 was the limit to maximum filesize (4Gb) which is easily exceeded with satellite imagery or virtual machines. NTFS, as I note above, is just painful for file permissions and isn’t really needed for an external drive which has hardware encryption. Well, as it turns out the answer was staring me in the face - exFAT which is essentially FAT64. OK, so its a proprietary Microsoft file system, but solves the filesize limit, is relatively simple and fast, ideally suited to external media and particularly flash drives, but perfectly suited to this use. Compatible with add-ons to Windows XP and MacOS, whilst there appears to be driver support for Linux. Given 99.9% of the time I move between Windows systems that suits me.

So, 500Gb of data transfer later I now have an operational drive. Just got to move the other backup disk from 500Gb to 1Tb…

I’m currently working on a pet project doing some timelapse (if you hadn’t guessed!) involved geocoding the photos (and a much longer blog on this process at a later date once it’s complete). However as part of this I needed to iterate over a series of files using the excellent nconvert. DOS batch files let you do this but the syntax (for me at least!) is a little confusing. So, in all it’s glory here’s the solution I rustled up after looking at various websites:

for /f %%a IN (’dir /b *.jpg’) do ..\nconvert -out jpeg -wmflag bottom-right -wmfile overlay\%%~na.png -o final\%%~na.jpg %%~na.jpg

As ever, much useful information can be gleaned by typing

for /?

The /f parses a list of files (denotes by the variable %%a) and this is fed to it by the ‘dir /b *.jpg’ command. In this case a “bare” directory listing of all JPG files. Once that is complete it runs the nconvert command inserting the variable at the %%a locations.

Crucially the statement is then finished with \%%~na.jpg which strips the file*name* from the 3-letter filetype (suffix). This is needed in the nconvert command.

One line of code, 1000 files processed very rapidly! Love it when this stuff works!!