Essential tools for manipulating big data files by Daniel Rubio.
From the post:
You can leverage several tools that are commonly used to manipulate big data files, which include: Regular expressions, sed, awk, WYSIWYG editors (e.g. Emacs, vi and others), scripting languages (e.g. Bash, Perl, Python and others), parsers (e.g. Expat, DOM, SAX and others), compression utilities (e.g. zip, tar, bzip2 and others) and miscellaneous Unix/Linux utilities (e.g. split, wc, sort, grep)
And,
10 Awesome Examples for Viewing Huge Log Files in Unix by Ramesh Natarajan.
Viewing huge log files for trouble shooting is a mundane routine tasks for sysadmins and programmers.
In this article, let us review how to effectively view and manipulate huge log files using 10 awesome examples.
cover the same topic but with very little overlap (only grep as far as I can determine).
Are there other compilations of “tools” that would be handy for large data files?