An example of why data will not end debate by Kaiser Fung.
From the post:
One oft-repeated “self-evident” tenet of Big Data is that data end all debate. Except if you have ever worked for a real company (excluding those ruled by autocrats), and put data on the table, you know that the data do not end anything.
Reader Ben M. sent me to this blog post by Benedict Evans, showing a confusing chart showing how Apple has “passed” Microsoft. Evans used to be a stock analyst before moving to Andreessen Horowitz, a VC (venture capital) business. He has over 25,000 followers on Twitter.
….
Evans responded to many of these comments by complaining that readers are not getting his message. That’s an accurate statement, and it has everything to do with the looseness of his data. This reminds me of Gelman’s statistical parable. The blogger here is not so much interested in how strong his evidence is but more interested in evangelizing the morale behind the story.
A highly entertaining post as always.
Gelman’s “statistical parable” is used for stories that cite numbers that if you think about them, are quite unreasonable. Gelman’s example case was statistics on death rates that put 1/4 of the death rate at a hospital as due to record keeping errors. Probably not true.
The point being that people bolster a narrative with numbers in the interest of advancing the story, with little concern for the “accuracy” of the numbers.
Other examples include: RIAA numbers on musical piracy, software piracy, OMB budget numbers, TSA terrorist threat numbers, etc.
I put “accuracy” in quotes because recognizing a “statistical parable” depends on where you sit. If you are on the side with shaky numbers, the question of accuracy is an annoying detail. If you oppose the side with shaky numbers, it is evidence they can’t make a case without manufactured evidence.
I take Kaiser’s point to be data is not magic. Even strong (in some traditional sense) data is not magic.
.
Data is at best one tool of persuasion that you can enlist for your cause, whatever that may be. Ignore other tools of persuasion at your own peril.
