Prison Polling by Carl Bialik.
From the post:
My print column examines the argument of a book out this week that major federal surveys are missing an important part of the population by not polling prisoners.
“We’re missing 1% of the population,” said Becky Pettit, a University of Washington sociologist and author of the book, “Invisible Men.” “People might say, ‘That’s not a big deal.’ “But it is for some groups, she writes — particularly young black men. And for young black men, especially those without a high-school diploma, official statistics paint a rosier picture than reality on factors such as employment and voter turnout.
“Because many surveys skip institutionalized populations, and because we incarcerate lots of people, especially young black men with low levels of education, certain statistics can look rosier than if we included” prisoners in surveys, said Jason Schnittker, a sociologist at the University of Pennsylvania. “Whether you regard the impact as ‘massive’ depends on your perspective. The problem of incarceration tends to get swept under the rug in lots of different ways, rendering the issue invisible.”
A reminder that assumptions are cooked into data long before it reaches us for analysis.
If we don’t ask questions about data collection, we may be passing on results that don’t serve the best interests of our clients.
So for population data, ask (among other things):
- Who was included/excluded?
- How were the included selected?
- On what basis were people excluded?
- Where are the survey questions?
- By what means were the questions asked? (phone, web, in person)
- Time of day of survey?
and I am sure there are others.
Don’t be impressed by protests that your questions are irrelevant or the source has already “accounted” for that issue.
Right.
When someone protests you don’t need to know, you know where to push. Trust me on that one.