Scientific Fraud and Systemic Barriers to Wider Research

Recently, a very instructive article by Dr. David Goodstein, a Vice Provost at the California Institute of Technology (aka Caltech), was brought to my attention on the topic of scientific fraud. I found the article to be extremely interesting and relevant because many (but by no means all) anti-circumcision activists (aka intactivists) are quick to claim the research by Auvert, Bailey and Gray et al is scientific fraud.

Oh how I wish it were.

But alas, I think it is most unhelpful for skeptics reflexively to make the accusation that the researchers, who may desperately want circumcision to prevent HIV, conducted their research in a fraudulent manner. Motive, we have to accept, is irrelevant if the science is done properly, as Dr. Goodstein points out.

The most stunning observation of Dr. Goodstein is that scientific fraud most often occurs in the biomedical field. He identifies three prerequisite conditions found in all cases of such fraud. While not always indicative of fraud, in the cases Dr. Goodstein has looked at, “three motives, or risk factors have always been present. In all cases, the perpetrators:

  1. Were under career pressure;
  2. Knew, or thought they knew what the answer would turn out to be if they went to all the trouble of doing the work properly, and
  3. Were working in a field where individual experiments are not expected to be precisely reproducible.”

Hugh Young points out that the relevant portion of Dr. Goodstein’s article to the studies this website is chiefly concerned about is the following:

Peer review is quite a good way to identify valid science.  It was wonderfully well suited to an earlier era when progress in science was limited only by the number of good ideas available.  Peer review is not at all well suited, however, to adjudicate an intense competition for scarce resources such as research funds or pages in prestigious journals.  The reason is obvious enough.  The referee, who is always among the few genuine experts in the field, has an obvious conflict of interest.  It would take impossibly high ethical standards for referees to fail to use their privileged anonymity to their own advantage, but, as time goes on, more and more referees have their ethical standards eroded by receiving unfair reviews when they are authors.  Thus the whole system of peer review is in peril.

Editors of scientific journals and program officers at the funding agencies have the most to gain from peer review, and they steadfastly refuse to believe that anything might be wrong with the system.  Their jobs are made easier because they have never had to take responsibility for decisions.  They are also never called to account for their choice of referees, who in any case always have the proper credentials.  Since the referees perform a professional service, almost always without pay, the primary responsibility of the editor or program officer is to protect the referee.  Thus referees are never called to account for what they write in their reviews.  As a result, referees are able, with relative impunity, to delay or deny funding or publication to their rivals.  When misconduct of this kind occurs, it is the referee who is guilty, but it is the editors and program officers who are responsible for propagating a corrupt system that makes misconduct almost inevitable.

This is the kind of misconduct that is, I fear, rampant in all fields of science, not only biomedical science.  …

Hugh Young adds this from Voodoo Science by Robert Park by way of example for Dr. Goodstein’s observations:

Few scientists were aware of the Wertheimer-Leeper work [claiming to show that children in houses with high electromagnetic fields were more likely to contract leukaemia] at the time, and fewer still took it seriously. In the first place, the study was not “blind”: she knew in advance which were the homes of leukemia victims. In the second place, the relative strength of the power-line fields was not actually measured but merely estimated on the basis of the size and proximity of power lines.

The situation was ripe for investigator bias; the tendency to judge the wiring of victims’ homes more critically would be almost unavoidable. If the result for a particular home disagreed with the researcher’s expectation, for example, there would be a tendency to double-check the result and see if something had been missed the first time. To the researchers, it may only seem that they are being careful, but unless all the homes are double-checked, it introduces a powerful bias. The numbers, after all, are very small — childhood leukemia is a rare disease—and the shift of only a few victims’ homes from “low field” to “high field” is sufficient to change the conclusion.

Scientists must constantly be on guard against this sort of self- deception. Unless studies are carefully designed to avoid it, the biases of the epidemiologist have a way of creeping into the results. To minimize the opportunity for bias, scientists rely on double-blind studies. An independent researcher might be given a list including both the homes of victims of childhood leukemia and an equal number of addresses of nonvictim children matched in age, gender, race, family income, etc., but without any indication of which are which. Without knowing which were the homes of victims and which were “controls,” the researcher would rate them by whatever criteria were used to estimate the field strength. Someone else would then apply the key after the judgments were made.

But even if the study had been double blind, a “risk ratio” of only three for a rare disease such as childhood leukemia would be regarded by many epidemiologists as barely credible. The risk ratio for lung cancer from smoking, for example, is well over thirty, that is, a 3,000 percent increase in the incidence of lung cancer among smokers. Yet it took years of checking and rechecking the figures, as well as a highly plausible mechanism in terms of known carcinogens in tobacco smoke and, finally, confirming laboratory studies on animals before the cancer link was firmly nailed down.

Obviously, easy parallels with the circumcision studies can be drawn. In a rampant disease environment, any small decrease can be misinterpreted and incorrectly attributed to a single factor under study when the true cause is likely more complex. Truly double blind studies are impossible for something like efficacy of circumcision in HIV risk reduction. Furthermore, there are just too many variables to attribute the small reductions in HIV infection to circumcision alone. Yet, we hear too often that the studies have proven beyond any doubt what circumcised American researchers and layman alike are already predisposed to believe, that circumcision reduces risk.

Editors note: The Goodstein article is available here. I apologize for any formatting errors in the quote from Robert Parker’s book. Likewise, for the lack of page numbers.

About David Wilton

fronterizo, public defender, intactivist, gay
This entry was posted in Culture, Medicine and tagged , , , , , , , , . Bookmark the permalink.