next up previous
Next: About this document Up: Basic Issues in Experiment Previous: Ecological Validity: Making Experiments

Conclusion

A single theme runs through this chapter: an experiment should tell us whether x (or several x's) truly influences y. It should not obscure the influence or suggest an influence where none exists. An experiment should be free of spurious effects and sampling biases. It should convince us that x and not something else influences y. It is difficult to design a controlled experiment, free of spurious effects and sampling biases, with a dependent variable that unambiguously represents a behavior of interest. Even if one gets this far, it is often hard to guess which experimental parameters--the values of independent variables and parameters of the experimental procedure--will most clearly demonstrate the effect of x on y. For this reason, researchers usually run pilot experiments. And all this machinery is pointless if one's experimental question isn't motivated by an interesting research question.

Now the bad news: even when x truly influences y, and one's experiment is perfectly designed and well motivated, the experiment might not demonstrate the influence. This is because x's influence on y is sometimes overwhelmed by the background noise of other influences. Separating the variance in y into components due to x and components due to everything else is the central task of statistical hypothesis testinghypothesis testing, the subject of the next chapter.



next up previous

Exper imental Methods for Artificial Intelligence, Paul R. Cohen, 1995
Mon Jul 15 17:05:56 MDT 1996