next up previous
Next: What is an Extraneous Up: Basic Issues in Experiment Previous: Basic Issues in Experiment

The Concept of Control

Like many little children, I had a chemistry set. One day, mixing water and copper sulfate, I produced a beautiful blue-green liquid with which I tried to wash away some of the grime on my chemistry bench. It worked and I was convinced I had invented a new cleanser. My dad, however, was unimpressed. He suggested I should try water alone to see whether it cleaned as well as my new solution. What an awesome revelation! I insisted that water alone absolutely wouldn't work, but this only drew the noose tighter around my neck, because if you need a control condition, insisting that you don't only makes you look more foolish.

Perhaps you think children make these mistakes but adults do not. Guess again! It can be very difficult to construct the right control condition, as a recent uproar in the AIDS research community attests. The following excerpt is from an article in the British weekly New Scientist:

The search for an AIDS vaccine was thrown into disarray last week with the disclosure of a ``stunning'' finding from experiments on monkeys carried out by Britain's Medical Research Council....

The MRC researchers gave four macaques a vaccine based on human T cells that had been infected with SIV [a virus related to HIV, which causes AIDS] and then inactivated. When they gave these monkeys live virus, three out of four were protected. But the shock came from four other monkeys. The researchers gave these animals uninfected human cells of the same type as those used to create the vaccine. These cells had never seen SIV. To the team's amazement, when they gave the animals live SIV, two of them were protected....

Some scientists were angry that the vital control experiment with uninfected cells had not been done earlier. But Jim Stott of the MRC countered that the need for such a control was not obvious at the beginning.... ``It's terribly easy to say that afterwards,'' he said. ``It would have been such a bizarre experiment to suggest. You have to try to save animals.'' (New Scientist, 21 September, 1991, p.14)

The New Scientist article is accompanied by a cartoon showing one macaque saying to another, ``Perhaps we should be doing the research?'' Let's examine what happened: scientists developed a vaccine with two components, human cells and virus. They garnered evidence to suggest the vaccine conferred immunity (three of four animals were protected). They assumed (with ample precedent from previous studies of vaccines) that the causative mechanism was a response to the virus, but in retrospect, they should have considered other hypotheses: The response could have been caused by the human cells and not the virus, or by human cells or the virus, or by an interaction between the agents. When it's spelled out this way, one cannot comprehend how the scientists--and their reviewers--simply assumed the virus was the causal agent and the T cells were not. Yet, as their spokesman said, ``It would have been such a bizarre experiment to suggest. Clearly, the threat to experimental control was not technical competence--the scientists were perfectly capable of designing appropriate control conditions--the threat arose from something more pernicious: Controlled experiments do not control all possible alternative explanations directly, they control all plausible alternatives, so if something isn't plausible, it is controlled only indirectly through random sampling. The threat to control in this case was the zeitgeist, the collective beliefs of the field, which made human T cells an implausible cause of immunity.

Technically, however, the lesson is this: Before you claim x causes y, you must rule out all other possible causes of y, called extraneous variables. An extraneous variable (ev) is any variable other than the independent variable x that affects the dependent variable y. The most basic controlled experiment comprises two conditions, called the control condition and the treatment condition, which are identical except for x. In particular, the extraneous variables in the treatment and control conditions are identical:

 Treatment Condition : x&tex2html_wrap_inline1237 

Control Condition : tex2html_wrap_inline1237

If tex2html_wrap_inline1241 , then x rather than tex2html_wrap_inline1245 influences y. However, we can't rule out the possibility that the effect tex2html_wrap_inline1249 is due to x acting in concert with one of the extraneous variables.

The story of the SIV experiment can be summed up this way:

 Treatment Condition : virus & human T cells tex2html_wrap_inline1253 immunity 

And here's the control condition, which was run much later:

 
Control Condition : human T cells tex2html_wrap_inline1253 immunity

Perhaps in the next round of experiments, human T cells will not be regarded as extraneous variables, but might be the focus, the hypothesized causal agent of immunity. I mention this to emphasize that the choice of independent variables is the researcher's choice. Nobody should tell researchers what causal factors to study, but nobody should listen to a researcher who asserts a causal association in the absence of an appropriate control condition.




next up previous
Next: What is an Extraneous Up: Basic Issues in Experiment Previous: Basic Issues in Experiment

Exper imental Methods for Artificial Intelligence, Paul R. Cohen, 1995
Mon Jul 15 17:05:56 MDT 1996