CSVed

Tuesday, September 28, 2010

I’ve blogged a couple of times about the utility of CSVs as a file format for the distribution of data. They are horribly inefficient, but incredibly simple and widely supported. For that reason they are good for distributing data and forever crop up on Wikileaks and Free Our Data. Excel usually grabs CSV as an extension, but its quite often easier to have something that’s designed for the job and CSVed does this admirably. You get a spreadsheet like view that allows in-cell editing, but its the many other touches that are nice. Drag-and-drop rows and columns, append files, change delimiters, split columns, prefix/suffix, filtering, sorting. The list is quite long and targeted at these dealing with large scale manipulation of CSVs.

CSVed comes highly recommended, but, like Excel there is the odd glitch. Its only with Excel 2010 that some of the file constraints are easing, but upto this point you are pretty much limited to 256 columns and 64,000 rows. I often end up with more than 256 columns when dealing with spectral data from a spectroradiometer (usually well over a thousand). Unfortunately CSVed hangs when trying to load these and Excel won’t load them, which usually leaves me using R to load the data and then transposing the file for further manipulation.