Evaluating the efficacy of intervention

The standard for reaching a verdict in civil trials is "the preponderance of the evidence." This is a valid standard to apply to evaluating the medical literature as well. Every published report should be given weight based on physicians' judgment of its reliability. Larger-scale studies should be weighted more heavily than smaller-scale studies, randomized trials more heavily than observation studies, rigorously designed trials more heavily than studies that may be biased. If there seems to be a relationship between a factor and an outcome, this does not necessarily imply that the factor caused the outcome. The association could be a result of chance variation between individuals. Statistical testing allows researchers to exclude chance as a likely cause of the relationship, but this is the only explanation ruled out by a significance test. The relationship could be a result of bias: bias in the selection of individuals for the study, bias in measurement of the factor or the outcome, or bias in differential loss to follow-up. A thorough analysis of the data is necessary to identify and exclude other possible explanations of the association. A government agency dealing with environmental regulations experimented with replacing administrative law judges with scientists. It was believed that scientists were better qualified to make the necessary technical evaluations. The experiment was not considered successful because, as one observer remarked, "Judges are used to having to reach a verdict within a short period of time based on whatever evidence is presented--scientists just can't seem to make decisions." This illustrates a similar difference between the role of the researcher and the role of the physician. The physician must decide the best treatment for each patient based only on whatever evidence is available. There is little question, however, what the patient would decide if allowed to choose between receiving the standard therapy today or waiting 5 years until conclusive scientific evidence has been obtained about whether an alternative treatment is better. Few patients would choose to wait. Yet the decisions a physician makes today will be criticized in 5 years, especially by those who never have had to make similar decisions themselves. Some decisions will turn out to be wrong when additional data are available, but physicians must make choices based on the best data currently available. The choice of medicine as a profession implies the acceptance of a life sentence to jury duty: the evidence will never stop accumulating, and the verdict must be continually reevaluated.