CS545 fall 2016

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.

====== Bias when using feature selection ======

When using feature selection you need to be very careful in how you evaluate your classifier.

Here's the wrong way of doing it:

<code python>
from PyML import *

# the wrong way of using feature selection

data = SparseDataSet('colon.data')
# distinguish between normal tissue and tissue affected by colon cancer
# data is available from:
# http://mldata.org/repository/data/viewslug/colon-cancer/

# create an instance of the RFE feature selection method
rfe = featsel.RFE()
# a feature selector's train method selects a subset of features
rfe.train(data)

results1 = SVM().stratifiedCV(data)
</code>

If you run this you will get a classifier with perfect accuracy.  Now let's do it the right way:

<code python>
# the right way to perform feature selection:
# feature selection is performed as part of training the classifier
data = SparseDataSet('colon.data')
results2 = composite.FeatureSelect(SVM(), featsel.RFE()).stratifiedCV(data)
</code>

CS545 fall 2016

User Tools

Site Tools

Page Tools