How often have your heard the claim this or that technology is “web scale?”
How big is “web scale?”
Visit http://www.worldwidewebsize.com/ to get an estimate of the size of the Web.
As of today, the estimated number of indexed web pages for Google is approximately 47 billion pages.
How does that compare, say to scholarly literature?
Would you believe 1 trillion pages of scholarly journal literature?
An incomplete inventory (Fig. 1), divided into biological, social, and physical sciences, contains 400, 200, and 65 billion pages, respectively (see supplemental data*).
Or better with an image:
I didn’t bother putting in the trillion page data but for your information, the indexed Web is < 5% of all scholarly journal literature.
Nor did I try to calculate the data that Chicago is collecting every day with 10,000 video cameras.
Is your app ready to step up to human scale information retrieval?
*Advancing science through mining libraries, ontologies, and communities by JA Evans, A. Rzhetsky. J Biol Chem. 2011 Jul 8;286(27):23659-66. doi: 10.1074/jbc.R110.176370. Epub 2011 May 12.
[…] Machine parsing/searching are absolute necessities if you want to create topic maps on a human scale. (Web Scale? Or do you want to try for human scale?) […]
Pingback by Semantic Parsing with Combinatory Categorial Grammars « Another Word For It — August 5, 2013 @ 10:19 am