# CS545 fall 2016

### Site Tools

assignments:assignment3

# Differences

This shows you the differences between two versions of the page.

 assignments:assignment3 [2016/09/19 12:20]asa [Part 1] assignments:assignment3 [2016/09/20 09:34]asa [Part 1] Both sides previous revision Previous revision 2016/09/20 09:34 asa [Part 1] 2016/09/20 09:33 asa [Part 2] 2016/09/19 12:20 asa [Part 1] 2016/09/19 12:19 asa [Part 1] 2016/09/19 12:12 asa [Part 3] 2016/09/19 10:09 asa 2016/09/15 14:55 asa [Part 3] 2016/09/15 14:53 asa [Grading] 2016/09/15 14:53 asa [Grading] 2016/09/15 14:53 asa [Grading] 2016/09/15 14:49 asa [Submission] 2016/09/15 14:45 asa 2016/09/15 14:44 asa 2016/09/15 14:43 asa 2016/09/15 12:53 asa 2016/08/09 10:25 external edit2015/10/15 12:24 asa 2015/10/09 13:11 asa 2015/10/09 13:10 asa [Submission] 2015/10/07 12:04 asa 2015/10/02 12:42 asa 2015/10/02 12:12 asa 2015/10/02 12:12 asa 2015/10/02 12:10 asa 2015/10/02 12:08 asa [Submission] 2015/10/02 12:05 asa 2015/10/02 10:04 asa 2015/10/02 09:48 asa Next revision Previous revision 2016/09/20 09:34 asa [Part 1] 2016/09/20 09:33 asa [Part 2] 2016/09/19 12:20 asa [Part 1] 2016/09/19 12:19 asa [Part 1] 2016/09/19 12:12 asa [Part 3] 2016/09/19 10:09 asa 2016/09/15 14:55 asa [Part 3] 2016/09/15 14:53 asa [Grading] 2016/09/15 14:53 asa [Grading] 2016/09/15 14:53 asa [Grading] 2016/09/15 14:49 asa [Submission] 2016/09/15 14:45 asa 2016/09/15 14:44 asa 2016/09/15 14:43 asa 2016/09/15 12:53 asa 2016/08/09 10:25 external edit2015/10/15 12:24 asa 2015/10/09 13:11 asa 2015/10/09 13:10 asa [Submission] 2015/10/07 12:04 asa 2015/10/02 12:42 asa 2015/10/02 12:12 asa 2015/10/02 12:12 asa 2015/10/02 12:10 asa 2015/10/02 12:08 asa [Submission] 2015/10/02 12:05 asa 2015/10/02 10:04 asa 2015/10/02 09:48 asa 2015/10/02 09:42 asa 2013/10/09 20:38 asa [Part 4: Using SVMs] 2013/10/09 12:19 asa 2013/10/06 20:54 asa 2013/10/06 20:51 asa 2013/10/06 15:23 asa 2013/10/06 15:23 asa 2013/10/06 15:22 asa 2013/10/06 13:33 asa 2013/10/06 11:54 asa 2013/10/06 11:54 asa 2013/10/06 11:52 asa 2013/10/04 21:08 asa [Part 3: Using the SVM] 2013/10/04 21:06 asa 2013/10/04 20:51 asa 2013/10/04 15:32 asa created Line 30: Line 30: With the code you just implemented,​ your next task is to explore the dependence of error on the value of the regularization parameter, $\lambda$. With the code you just implemented,​ your next task is to explore the dependence of error on the value of the regularization parameter, $\lambda$. - In what follows set aside 30% of the data as a test-set, and compute the in-sample error, and the test-set error as a function of the parameter $\lambda$ on the red wine data.  Choose the values of $\lambda$ on a logarithmic scale with values 0.01, 0.1, 1, 10, 100, 1000 and plot the RMSE only. + In what follows set aside 30% of the data as a validation-set, and compute the in-sample error, and the validation-set error as a function of the parameter $\lambda$ on the red wine data.  Choose the values of $\lambda$ on a logarithmic scale with values 0.01, 0.1, 1, 10, 100, 1000 and plot the RMSE. - Repeat the same experiment where instead of using all the training data, choose 20 random examples out of the training set, and train your model using those 20 examples. + Repeat the same experiment where instead of using all the training data, choose 20 random examples out of the training set, and train your model using those 20 examples, while evaluating on the same validation set. Now answer the following: Now answer the following: Line 43: Line 43: Regression Error Characteristic (REC) curves are an interesting way of visualizing regression error as described Regression Error Characteristic (REC) curves are an interesting way of visualizing regression error as described in the following [[http://​machinelearning.wustl.edu/​mlpapers/​paper_files/​icml2003_BiB03.pdf|paper]]. in the following [[http://​machinelearning.wustl.edu/​mlpapers/​paper_files/​icml2003_BiB03.pdf|paper]]. - Write a function that plots the REC curve of a regression method, and plot the REC curve of the best regressor you found in Part 1 of the assignment. + Write a function that plots the REC curve of a regression method, and plot the REC curve of the best regressor you found in Part 1 of the assignment ​(i.e. the one that gave the lowest error on the validation set).  Plot the REC curve for both the validation set and the training set. What can you learn from this curve that you cannot learn from an error measure such as RMSE or MAD? What can you learn from this curve that you cannot learn from an error measure such as RMSE or MAD?