20 Big Data Repositories You Should Check Out by Vincent Granville.
Vincent lists some additional sources along with a link to Bernard Marr’s original selection.
One of the issues with such lists is that they are rarely maintained.
For example, Bernard listed:
Topsy http://topsy.com/
Free, comprehensive social media data is hard to come by – after all their data is what generates profits for the big players (Facebook, Twitter etc) so they don’t want to give it away. However Topsy provides a searchable database of public tweets going back to 2006 as well as several tools to analyze the conversations.
But if you follow http://topsy.com/, you will find it points to:
Use Search on your iPhone, iPad, or iPod touch
With iOS 9, Search lets you look for content from the web, your contacts, apps, nearby places, and more. Powered by Siri, Search offers suggestions and updates results as you type.
That sucks doesn’t it? Expecting to be able to search public tweets back to 2006, along with analytical tools and what you get is a kiddie guide to search on a malware honeypot.
For a fuller explanation or at least the latest news on Topsy, check out: Apple shuts down Twitter analytics service Topsy by Sam Byford, dated December 16, 2015 (that’s today as I write this post).
So, strike Topsy off your list of big data sources.
Rather than bare lists, what big data needs is a curated list of big data sources that does more than list sources. Those sources need to be broken down to data sets to enable big data searchers to find all the relevant data sets and retrieve only those that remain accessible.
Like “link checking” but for big data resources. Data Source Checking?
That would be the “go to” place for big data sets and as bad as I hate advertising, a high traffic area for advertising to make it cost effective if not profitable.