Many positive scientific effects published in the literature seem to diminish in their significance with time; this is known as the decline effect. For example, initial parapsychological research indicated evidence for psychic ability, but this effect declined with subsequent studies. Some scientists link this to the strange statistical effect called regression to the mean: the phenomenon that if a variable is extreme on its first measurement, by the second measurement it will be closer to the average. It is impossible to test this, however, without access to negative results of scientific studies.
Most negative results of scientific experiments remain unpublished. This is a pervasive problem the world over, and is likely to be skewing the available scientific data pertaining to certain phenomena. When it turned out Eli Lilly (the makers of antidepressants like Prozac) had withheld the results of about a third of the trials conducted to win government approval, the clinical community was up in arms. Unfortunately, this is par the course for contemporary science.
Why is this occurring? One possible reason is that companies have a vested interest in publishing only positive results for their products, be they drugs, medical devices or shampoo. This is likely to be the case with Prozac. After all, if you can show your drug to have a great positive body of evidence supporting it then doctors and patients are more likely to prescribe it/buy it. Another possibility is that researchers simply feel like they have failed if they produce negative results. If you have spent years designing a study, carefully selecting your variables, experimental design and hypotheses, getting your hopes up that you might discover a new effect, then a set of negative results may come as a crushing blow.
The problem of course is that if the available evidence seems overwhelmingly positive then scientists, policy-makers and the general public take this to mean that there is an actual effect that exists in some real sense. Science is trying to discover the truth about the external world; positive results indicate evidence for an effect, therefore the effect exists in truth. Popper would of course have something to say about this conception of science, but it seems to be more akin to the lay conception and more relevant to how science actually seems to work.
It is of paramount importance, then, that we begin to see negative results in a different light, given how their absence from publications is skewing our perceptions of certain effects and – in the case of Prozac – potentially harming our health.
Related articles:
The Philosopher's Eye 