The Guardian: Data Visualisation 101

The Guardian have been forging ahead with the whole “data journalism” area (and I guess I should mention their book!!) and have been doing a remarkable job in ferreting out interesting datasets, finding links/stories (and lets face it, The Telegraph’s MP expenses story doesn’t really get any bigger, and it was all data), developing online visualisations and making the data available. It’s a fascinating area and one the deserves to be developed much more extensively.

Two recent columns are worth highlighting: firstly the Open Data Weekend which features a couple of useful presentations and then a slightly longer, more focused, write-up of the online services they use for visualisation. Well worth a read.

Podcasts for learning

Donald Clark wrote a good blog entry on the usefulness of audio, listening and podcasts for learning. Not least the ability to rapidly re-review, take notes and at faster playback. This all makes sense to me and I like catching up on audio shows, as well as TED videos, when on the move.

And a smartphone makes a lot of sense here in that you can subscribe to RSS feeds and download audio whilst on the move. You can even play back audio whist it is still downloading. On Android there are quite a few choices with BrightHub providing a good review although its a little out-of-date. Google Listen stands out for its search capabilities and integration with Google Reader. However it would appear adding the feature for increasing the play speed (a real time saver!) is more difficult for whatever technical reason. So much so that there is now a third party sound library that has this function called Presto. Several apps have added support for this feature and the Presto page lists some of these. I have been trying the current Beta (and free) ReadItOut Audiobook Player which adds the Presto library. If you just want something very lightweight and simple then you can’t go wrong with either Podcast Player or Tiny Player.

Photos that make you say “OMG!”

A great selection of historic photos that just shout “O MY GOD”. Fantastic!

Links roundup

A few links to follow up:

1. Pointclouds.org.uk: I don’t normally plug other institutions, but this site outlines a link-up between Faro and UCL and, basically, scanning a few locations to try to get on the BIM bandwagon. Its fun to play with the pointclouds though and see the direction people area headed.
2. Britain from Above: summary page of the English Heritage project to scan ~100K aerial photos dating from 1919. Due to go live this year and should be a wonderful historic resource. Watch this space.
3. Astun Loader: Astun have released Loader which takes the underlying OGR and allows easy conversion of GML/KML to other formats. Specifically designed to focus upon loading Ordnance Survey GML in to other formats/databases. useful.
4. TanDEM-X: TanDEM-X has now imaged the entire planet with good, medium and poor data. Mission well on track and results looking, well, exciting!

GEO-12 roundup

I attend the GEO12 trade show yesterday….. it’s hardly the most salubrious location (Holiday Inn in Elstree) but the space is reasonable and, well, its free and yes, you do get a free lunch!! More than that, it is doing a good job of combining the GIS and survey world.

Building Information Management (BIM) was the focus for the first session…. not the most exciting topic, but I can see this has the potential to transform building management from design, through build, delivery and maintenance. Key area to watch and there is plenty of money in the sector to make it work (and make it save money).

For me, the “fun” topics close to my interests were in the remote sensing. UAVs were a big focus at the show with both SeneseFly and Gatewing in attendance. Presentations were good from both although pizazz goes to SenseFly…. they went from packed to ready-to-launch with engine running in the conference room in 30 seconds!!! (I’ve blogged before on Gatewing). Some of the key aspects were:

  • total weight and payload. SenseFly total weight is under 500g to simplify licensing, Gatewing is 2kg with research looking at larger payloads
  • accuracy is an issue with both UAVs flying compact cameras. The Gatewing is 10MP but both resolution and camera quality limit accuracy. The ability to fly a DSLR would significantly improve imagery
  • the key point is software. Both offer orthphoto and DEM processing, but didn’t really explain how they did it (one assumes SfM type processing, possibly involving open-source libraries). We are getting close to the stage where these are mature products - yes, payload and cost are issues, but expect these to drop dramatically over the coming years
  • licensing - really key issue. These come under CAA regulations and require licensing as unmanned aerial vehicles flown under line-of-sight. Apparantly this is 500m horizontally and 400ft vertically (not sure why Gatewing switched from metric to imperial!). They have an extended license allowing them to operate at upto 750m

<

p>Its horses for courses when it comes to aerial systems: my biggest complaint with UAVs is payload, licensing and cost. These demos do nothing to alleviate these problems particularly, but things are headed in the right direction.

The final company was Spheron who specialise in full 306 degree imaging using a high dynamic range camera. In short, an SLR will take images with a small dynamic range (read radiometric resolution), this is 8-bit or about 5-10 f-stops. The human eye can see 18-20 f-stops. The Spheron is 32-bit at upto 28 f-stops. This allows you to image things across all contrast levels that are simply not possible with traditional cameras. Applications?? Two main areas:

  • Computer rendering: place the camera at a location in a real photo and image the scene. Use the light information to accurate render a computer model in the scene. The results are stunning
  • Asset Management: image different locations allowing you to get full 360 degree HDR imagery of assets.

The camera can operate in stereo (vertical offset), but currently is limited to manual, single, point measurements. An automated process is under investigation.

Pedagogic History

Donald Clark has started a blog marathon looking at a history of pedagogic practise. Its a self-selected list and will inevitably not include those you think it could or should….. but if you want to get a flavour and feel for the scope and extent then it makes fascinating reading. And in good academic style, there are plenty of references to follow up on.

A must read.

Testing 3D landform quantification methods with synthetic drumlins in a real digital elevation model

Hillier, J.K. and Smith, M.J. (2012)
Geomorphology


Metrics such as height and volume quantifying the 3D morphology of landforms are important observations that re?ect and constrain Earth surface processes. Errors in such measurements are, however, poorly under-stood. A novel approach, using statistically valid ‘synthetic’ landscapes to quantify the errors is presented. The utility of the approach is illustrated using a case study of 184 drumlins observed in Scotland as quanti?ed from a Digital Elevation Model (DEM) by the ‘cookie cutter’ extraction method. To create the synthetic DEMs, observed drumlins were removed from the measured DEM and replaced by elongate 3D Gaussian ones of equivalent dimensions positioned randomly with respect to the ‘noise’ (e.g. trees) and regional trends (e.g. hills) that cause the errors. Then, errors in the cookie cutter extraction method were investigated by using it to quantify these ‘synthetic’ drumlins, whose location and size is known. Thus, the approach deter-mines which key metrics are recovered accurately. For example, mean height of 6.8 m is recovered poorly at 12.50.6 (2s) m, but mean volume is recovered correctly. Additionally, quantification methods can be compared: A variant on the cookie cutter using an un-tensioned spline induced about twice (1.79) as much error. Finally, a previously reportedly statistically significant (p=0.007) difference in mean volume between sub-populations of different ages, which may reflect formational processes, is demonstrated to be only 30-50% likely to exist in reality. Critically, the synthetic DEMs are demonstrated to realistically model parameter recovery, primarily because they are still almost entirely the original landscape. Results are insen-sitive to the exact method used to create the synthetic DEMs, and the approach could be readily adapted to assess a variety of landforms (e.g. craters, dunes and volcanoes).

‘The Physical Landscape of Britain and Northern Ireland’: a project to increase geomorphological awareness

Hooke, J.M. and Smith, M.J.
Scottish Geographical Journal


A project has been initiated that will produce an interactive website searchable by maps and database terms, aimed at increasing knowledge and awareness of the geodiversity, landforms and processes of Britain and Northern Ireland,. It has the following objectives: (1) to make existing knowledge on the physical landscape and geomorphology of Britain more widely accessible; (2) to increase knowledge and awareness of physical attributes of the landscape; (3) to provide a synthesis of present knowledge; (4) to identify gaps in present knowledge as a platform for future societally relevant research. The components and steps in the project are outlined.

Leaves in Near infra-red

During my course on Remote Sensing last year I was trying to drive home to the students that EM radiation is comprised of different wavelengths and that Earth surface features reflect radiation very differently at different wavelengths. Vegetation is a nice example to use because of the dramatic change of reflectance from red to near infra-red (NIR). And you can show this even better by imaging vegetation they are familiar with.

Using my Nikon D70 which is particularly sensitive to NIR I photographed a dead leaf and live leaf in red, green and NIR (below). The postcard (click on it to get a hi-res version) shows each “band” as a greyscale image and you can clearly see the low reflectance in red and green and high reflectance (white) in NIR. The dead leaf still reflects more in NIR, but the difference is far less pronounced; indeed reflectance is nearly equal across all three bands which is why the leaf appears brown in the false-colour composite (bottom right) whilst the live leaf is red.

Freemind location

I blogged on Freemind, the open source mind mapping software, a few days ago, but wanted to add on to that post. The development team are reaching a v1.0 release candidate (currently in Beta) and have incorporated the ability to add location to nodes on your mind map. Tools -> Map (or Ctrl-Alt-M) brings up the Map window which loads OSM data of Europe. Type in the “Search Location” bar the place you want to add and hit enter. The location pointer is centred at this location. You can now:

  • zoom in to look at this in more detail.
  • move the point - or rather, single-click anywhere in the new window to place the pointer somewhere else
  • click Actions -> Place Selected Nodes to add the location to you node
  • double-click to place a new node in your mind map at that location

All very useful and interesting functions and good to see OSM integrating and enabling such a feature. It will be interesting to see how this develops.

Tax move?

Nice graphic over at The Guardian displaying historic tax rates across the world…. so, where do you want to move??

Reforming teaching and learning

Garr Reynolds has combined together a nice selection of contemporary videos on how teaching and learning could happen, the malaise in some areas of education and some of the suggestions for solutions. Ken Robinson’s cartoon talk is perhaps the most academic and well founded, but the others all add very interesting aspects, even if you dont agree with them! Well worth checking out and, it goes without saying, that it is universities that come in for the most severe treatment in terms of large lecture theatres and sterile content. However judge for yourself.

Happiness and Productivity

A great talk from Shawn Achor on TED.com detailing how happiness leads to productivity, rather than the western view of productivity leading to happiness (aka work hard, earn more, be happy). Its a little bit of a sales pitch but the summary of what this all means appears at 11.25.

There is quite a bit of science that goes behind this work….. interestingly Paul Ekman (Emotions Revealed) notes that you feel an emotion by creating the facial expression for it. So as made as it might seem (in our inhibited western society), Laughter Yoga (enforced laughter!) does have relevance in creating feelings of well being which develops in to a positive feedback mechanism. Thats not to say its the only way of doing this, and Shawn Achor has a list of other suggestions. Also worth noting that its a very funny talk!!

Freemind: great mind mapping software

I’m a big fan of mind mapping and whilst Tony Buzan is a bit over-the-top on the sales pitch (for a far more eloquent and scientifically founded piece on memory training read “Moonwalking with Einstein”), mind mapping is a great way of rapidly getting ideas down on paper and storing information in a condensed, and easily accessible, form. Its not linear which makes it quick to integrate.

There are some good commercial mind-mapping programs, but by far the best open-source product is Freemind. Its a Java app, cross-platform and very slick. OK, its not quite as Windows-centric as other applications, but its good and does everything you want, at least initially. Give it a try because it won’t disappoint you.

Docear: reference organisation, management and document outlining

http://www.youtube.com/watch?v=jRHqLktIMWw&feature=player_embedded

Geotagging - always a good idea?

Nice pick-up from The Guardian….. geotagging photos and uploading is a popular pastime for some, however it can have unintended consequences as the US Army points out.

Warren cited a real-world example from 2007. When a new fleet of helicopters arrived with an aviation unit at a base in Iraq, some Soldiers took pictures on the flightline, he said. From the photos that were uploaded to the Internet, the enemy was able to determine the exact location of the helicopters inside the compound and conduct a mortar attack, destroying four of the AH-64 Apaches.

Ouch! There must have been a big sigh when that happened. Might as well have stood there waving a flag saying shoot me.

Landsat 5 still refuses to die!!

Interesting news article over at USGS about their continued efforts to revive Landsat 5 (or more specifically the Thematic Mapper instrument). All hope is not (yet!) lost, but what’s more interesting is that if improvement in the transmission equipment (the reason for the failure) is not solved, there is a certain amount of life left in the kit which will be prioritized. At that point….. they will try to turn on the Multi-Spectral Scanner!! Yes, seriously. Landsat 5 was the last Landsat to have both TM and MSS (as a back-up) and this has been off, well, for a long-time. It would be very interesting to know the last MSS scene imaged. A while back me-thinks. And if they do get it, then (as they note), they need to re-instate all the old MSS workflows. Probably long forgotten and not designed to work within current IT systems. There is something “boys-own” about the whole affair which makes it all rather endearing…. that said, there is a serious side to all this. LDCM is not due for launch until January 2013 which currently gives a relatively large whole in Landsat data collection (albeit with the limited capacity of Landsat 7).

Google adds postcode extents

Nice pick-up by The Guardian….. Google Maps now defines postcode extent when you query the postcode itself: for example. I’m assuming this is CodePoint, as per my previous blog.

Las Vegas: a Landsat Timeline



This shows the real power of the Landsat archive….watch a desert transform in to a city over 40 years. The ability for remote sensing to offer such a relatively simple, but powerful, output is remarkable. It also offers a visual critique of modern society…

Happy birthday Envisat

Envisat’s birthday was nicely flagged by NASA EOID….at 10 years old it is doing pretty well and is required to remain in service until the Sentinel range of satellites (part of the GMES programme) become fully operational. Envisat is a big bugger of a satellite….. claimed to be the largest ever EO satellite launched (26m long) it weighs in at an impressive 8.5t and carries 10 science instruments, including the flagship radar ASAR and super-spectral MERIS (useful summary at Wikipedia).