Splitting a Large CSV File into Separate Smaller Files Based on Values Within a Specific Column by Tony Hirst.
From the post:
One of the problems with working with data files containing tens of thousands (or more) rows is that they can become unwieldy, if not impossible, to use with “everyday” desktop tools. When I was Revisiting MPs’ Expenses, the expenses data I downloaded from IPSA (the Independent Parliamentary Standards Authority) came in one large CSV file per year containing expense items for all the sitting MPs.
In many cases, however, we might want to look at the expenses for a specific MP. So how can we easily split the large data file containing expense items for all the MPs into separate files containing expense items for each individual MP? Here’s one way using a handy little R script in RStudio…
Just because data is “open,” doesn’t mean it will be easy to use. (Leaving the useful question to one side.)
We have been kicking around idea for a “killer” topic map application.
What about a plug-in for a browser that recognizes file types and suggests tools for processing them?
I am unlikely to remember this post a year from now when I have a CSV file from some site.
But if a browser plugin recognized the extension, .csv, and suggested a list of tools for exploring it….
Particularly if the plug-in called upon some maintained site of tools, so the list of tools is maintained.
Or for that matter, that it points to other data explorers who have examined the same file (voluntary disclosure).
Not the full monty of topic maps but a start towards collectively enhancing our experience with data files.