Most scientists who are honest about their work realize that they are wrong the majority of their scientific careers. Wrong in that science in general (and medical science in particular) is full of blind alleys that lead nowhere but consume years of one’s time. Wrong in that we may measure that which is unimportant and assume it is important because we can measure it. Wrong in that we are misled by early successes involving small numbers only to be later chastised by what I call the God of Large Numbers. Wrong because we are humans and therefore prey to all the usual human failures: vanity, dogmatism, and bias.

This wrongness is the subject of two recent books which you might consider reading: Wrong by David H. Freedman, and Being Wrong by Kathryn Schulz. Though they both take on the same general subject, they have very different takes on wrongness, as a recent New York Times review of the two suggests. Freedman basically thinks that the entire idea of expertise is flawed, given the frequency with which experts are wrong in their major conclusions, and that medical progress is largely illusory. Schulz views being wrong as a very human tool that brings us ever closer to the truth: a good thing.

These works make me think about the endless scientific sessions I’ve sat through over the years. In the average ASCO session, say with eight to ten presentations, I may come home with one nugget of information that changes my practice. Freedman would say that that nugget, as often as not, is fool’s gold, and I cannot disagree. Many of the things I have learned had to be later unlearned. But to conflate the erroneous nature of a particular study with the overall scientific process is equally mistaken. A favorite statement in financial circles is that the stock market is a voting machine in the short run, and a weighing machine in the long run. The same is true of the scientific process. We are in it for the long run; we deal in marathons rather than sprints.

And it is more than just whether a particular bit of data is right or wrong, of course. Something can be right scientifically and quite useless, or arguably of marginal benefit. Medicine is full of value judgments, because ultimately we care about more than just simple mortality statistics. A hip replacement may not prolong the life of a seventy-year-old, but may keep that elderly man mobile. Closer to home (because I have been intimately involved in the area for the past decade), anti-VEGF therapy prolongs progression-free survival in three randomized controlled trials in breast cancer, yet fails to prolong overall survival. Should it be used? Here we enter a thicket of value judgments: what is the cost of the therapy and does it represent good value? What side effects is the patient exposed to? Does prolonging time to progression improve quality of life or reduce suffering to any significant degree (surprisingly difficult to measure, or to know what the measurements mean)?

All of which is to say that this isn’t easy. I’ll end this blog with a fragment of a poem by John Donne (from his Satire III), which tells you all you need to know:

...though truth and falsehood be
Near twins, yet truth a little elder is;
Be busy to seek her; believe me this,
He's not of none, nor worst, that seeks the best.

To adore, or scorn an image, or protest,

May all be bad; doubt wisely; in strange way
To stand inquiring right, is not to stray;

To sleep, or run wrong, is. On a huge hill,
Cragged and steep, Truth stands, and he that will
Reach her, about must and about must go,
And what the hill's suddenness resists, win so.
Yet strive so that before age, death's twilight,

Thy soul rest, for none can work in that night.
To will implies delay, therefore now do;

Hard deeds, the body's pains; hard knowledge too
The mind's endeavours reach, and mysteries

Are like the sun, dazzling, yet plain to all eyes.

Keep the truth which thou hast found.