In a recent story about randomized medical trials:
“I should leave the final word to Archie Cochrane. In his trial of coronary care units, run in the teeth of vehement opposition, early results suggested that home care was at the time safer than hospital care. Mischievously, Cochrane swapped the results round, giving the cardiologists the (false) message that their hospitals were best all along.
“They were vociferous in their abuse,” he later wrote, and demanded that the “unethical” trial stop immediately. He then revealed the truth and challenged the cardiologists to close down their own hospital units without delay. “There was dead silence.”
Followed by Harford’s closing line: “The world often surprises even the experts. When considering an intervention that might profoundly affect people’s lives, if there is one thing more unethical than running a randomised trial, it’s not running the trial”
One of the persistent dangers of randomized trials is that the results can contradict what is “known” to be true by experts.
Another reason for user rather than c-suite “testing” of product interfaces, assuming the c-suite types are willing to hear “bad” news.
And a good illustration that claims of “ethics” can be hiding less pure concerns.
I first saw this in A brilliant anecdote on how scientists react to science against their interests by Chris Blattman, which lead me to: Weekly Links May 2: Mobile phones, working with messy data, funding, working with children, and more… and thence to the original post: The random risks of randomised trials by Tim Harford.