One goal of methods is to facilitate the independent verification of scientific observations. Thus, many experimental techniques-such as statistical tests of significance, double-blind trials, or proper phrasing of questions on surveys-have been designed to minimize the influence of individual bias in research. By adhering to these techniques, researchers produce results that others can more easily reproduce, which promotes the acceptance of those results into the scientific consensus.


If research in a given area does not use generally accepted methods, other scientists will be less likely to accept the results. This was one of several reasons why many scientists reacted negatively to the initial reports of cold fusion in the late 1980s. The claims were so physically implausible that they required extraordinary proof. But the experiments were not initially presented in such a way that other investigators could corroborate or disprove them. When the experimental techniques became widely known and were replicated, belief in cold fusion quickly faded.

In some cases the methods used to arrive at scientific knowledge are not very well defined. Consider the problem of distinguishing the “facts” at the forefront of a given area of science. In such circumstances experimental techniques are often pushed to the limit, the signal is difficult to separate from the noise, unknown sources of error abound, and even the question to be answered is not well defined. In such an uncertain and fluid situation, picking out reliable data from a mass of confusing and sometimes contradictory observations can be extremely difficult.

In this stage of an investigation, researchers have to be extremely clear, both to themselves and to others, about the methods being used to gather and analyze data. Other scientists will be judging not only the validity of the data but also the validity and accuracy of the methods used to derive those data. The development of new methods can be a controversial process, as scientists seek to determine whether a given method can serve as a reliable source of new information. If someone is not forthcoming about the procedures used to derive a new result, the validation of that result by others will be hampered.

Methods are important in science, but like scientific knowledge itself, they are not infallible. As they evolve over time, better methods supersede less powerful or less acceptable ones. Methods and scientific knowledge thus progress in parallel, with each area of knowledge contributing to the other.

A good example of the fallibility of methods occurred in astronomy in the early part of the twentieth century.

One of the most ardent debates in astronomy at that time concerned the nature of what were then known as spiral nebulae-diffuse pinwheels of light that powerful telescopes revealed to be quite common in the night sky.

Some astronomers thought that these nebulae were spiral galaxies like the Milky Way at such great distances from the earth that individual stars could not be distinguished. Others believed that they were clouds of gas within our own galaxy.

One astronomer who thought that spiral nebulae were within the Milky Way, Adriaan van Maanen of the Mount Wilson Observatory, sought to resolve the issue by comparing photographs of the nebulae taken several years apart. After making a series of painstaking measurements, van Maanen announced that he had found roughly consistent unwinding motions in the nebulae. The detection of such motions indicated that the spirals had to be within the Milky Way, since motions would be impossible to detect in distant objects.

Van Maanen's reputation caused many astronomers to accept a galactic location for the nebulae. A few years later, however, van Maanen's colleague Edwin Hubble, using the new 100-inch telescope at Mount Wilson, conclusively demonstrated that the nebulae were in fact distant galaxies; van Maanen's observations had to be wrong. Studies of van Maanen's procedures have not revealed any intentional misrepresentation or sources of systematic error. Rather, he was working at the limits of observational accuracy, and his expectations influenced his measurements.

Though van Maanen turned out to be wrong, he was not ethically at fault. He was using methods that were accepted by the astronomical community as the best available at the time, and his results were accepted by most astronomers. But in hindsight he relied on a technique so susceptible to observer effects that even a careful investigator could be misled.

The fallibility of methods is a valuable reminder of the importance of skepticism in science. Scientific knowledge and scientific methods, whether old or new, must be continually scrutinized for possible errors. Such skepticism can conflict with other important features of science, such as the need for creativity and for conviction in arguing a given position. But organized and searching skepticism as well as an openness to new ideas are essential to guard against the intrusion of dogma or collective bias into scientific results.

–> 2.2 Values and Judgment in Science (i)