How to do Science

Ok, that's rather an ambitious title. I will, in fact, only cover a bit of how science is done. But one of my main themes here is how science is done, what the thinking is like, what the language is like and why. I would like to convey a better understanding of what scientists do and how to understand what they have to say.

For today's discussion, I want to start with a really excellent essay Richard Feynman wrote, based on a commencement address he gave in 1974. Feynman was one of the nation's leading physicists from the end of WWII until his death in 1988. He was a Nobel prize winner and was perhaps most famous for uncovering the problems with the o-rings that were responsible for the Challenger disaster.

Read the whole essay, but there is one part in particular that I want to discuss here. At this point Feynman is talking about one central requirement for doing good science.

That is the idea that we all hope you have learned in studying science in school -- we never say explicitly what this is, but just hope that you catch on by all the examples of scientific investigation. It is interesting, therefore, to bring it out now and speak of it explicitly. It's a kind of scientific integrity, a principle of scientific thought that corresponds to a kind of utter honesty -- a kind of leaning over backwards. For example, if you're doing an experiment, you should report everything that you think might make it invalid -- not only what you think is right about it: other causes that could possibly explain your results; and things you thought of that you've eliminated by some other experiment, and how they worked -- to make sure the other fellow can tell they have been eliminated.

Details that could throw doubt on your interpretation must be given, if you know them. You must do the best you can -- if you know anything at all wrong, or possibly wrong -- to explain it. If you make a theory, for example, and advertise it, or put it out, then you must also put down all the facts that disagree with it, as well as those that agree with it. There is also a more subtle problem. When you have put a lot of ideas together to make an elaborate theory, you want to make sure, when explaining what it fits, that those things it fits are not just the things that gave you the idea for the theory; but that the finished theory makes something else come out right, in addition.

In summary, the idea is to give all of the information to help others to judge the value of your contribution; not just the information that leads to judgment in one particular direction or another.

It is certainly true that scientists do not always live up to this ideal of presenting all critical evidence. Scientists are human too and sometimes fail to live up to the highest ideals. However, doing good science and gaining the greatest reputation in science requires following this ideal as closely as at all possible. Generally speaking science follows the above ideals pretty well.

This requirement of presenting contradictory evidence, however, is not typical in most other activities. Neither a lawyer not an advertiser nor a salesman would do too well if he or she spent too much time pointing out the possible flaws in his clients case. Because of this characteristic the way that scientists talk about their work often seems a bit strange to non-scientists.

In almost all areas of science there exists some contradictory evidence, or at least evidence which doesn't fit the current theories as well as we would like. So there is almost always some contradictory evidence to recognize and therefore some uncertainty in our conclusions. But also realize that in some cases there is very little that doesn't match up just right, and even then the mismatch is small, and in other cases there are large puzzling problems and other cases that fall in every degree of uncertainty between these extremes. Therefore one of the striking characteristics of scientific speech is an effort to express the degree of uncertainty. Hence the frequent use of qualifiers and so called "weasel" words.

See, in most non-scientific venues people want to come to some conclusion, even if they are not absolutely certain it is correct, and yet act upon it with some action in spite of any uncertainty. In those circumstances "weasel" wording is undesirable. In the scientific arena, they are an essential part of communicating the degree of uncertainty and are necessary for reaching the best, most likely to be correct, conclusions that we can reach.