On Being a Data Skeptic by Cathy O’Neil. (pdf)
From Skeptic, Not Cynic:
I’d like to set something straight right out of the gate. I’m not a data cynic, nor am I urging other people to be. Data is here, it’s growing, and it’s powerful. I’m not hiding behind the word “skeptic” the way climate change “skeptics” do, when they should call themselves deniers.
Instead, I urge the reader to cultivate their inner skeptic, which I define by the following characteristic behavior. A skeptic is someone who maintains a consistently inquisitive attitude toward facts, opinions, or (especially) beliefs stated as facts. A skeptic asks questions when confronted with a claim that has been taken for granted. That’s not to say a skeptic brow-beats someone for their beliefs, but rather that they set up reasonable experiments to test those beliefs. A really excellent skeptic puts the “science” into the term “data science.”
In this paper, I’ll make the case that the community of data practitioners needs more skepticism, or at least would benefit greatly from it, for the following reason: there’s a two-fold problem in this community. On the one hand, many of the people in it are overly enamored with data or data science tools. On the other hand, other people are overly pessimistic about those same tools.
I’m charging myself with making a case for data practitioners to engage in active, intelligent, and strategic data skepticism. I’m proposing a middle-of-the-road approach: don’t be blindly optimistic, don’t be blindly pessimistic. Most of all, don’t be awed. Realize there are nuanced considerations and plenty of context and that you don’t necessarily have to be a mathematician to understand the issues.
….
It’s a scant 26 pages, cover and all but “On Being a Data Skeptic” is well worth your time.
I particularly liked Cathy’s coverage of issues such as: People Get Addicted to Metrics, which ends with separate asides to “nerds,” and “business people.” Different cultures and different ways of “hearing” the same content. Rather than trying to straddle those communities, Cathy gave them separate messages.
You will find her predator/prey model particularly interesting.
On the whole, I would say her predator/prey analysis should not be limited to modeling. See what you think.