Assignment 5: Neural networks

Due: October 31st at 11:59pm

Part 1: Multi-layer perceptrons

In the first few slides about neural networks (also section 7.1 in chapter e-7) we discussed the expressive power of multi-layer perceptrons with a “sign” activation function. Describe in detail a multi-layer perceptron that implements the following decision boundary:

Part 2: Exploring neural networks for digit classification

In this segment of the assignment we will explore classification of handwritten digits with neural networks. For that task, we will use part of the MNIST dataset, which is very commonly used in the machine learning community. Your task is to explore various aspects of multi-layer neural networks using this dataset. We have prepared a small subset of the data with given split into training and test data.

Here's what you need to do:


Submit your report via Canvas. Python code can be displayed in your report if it is short, and helps understand what you have done. The sample LaTex document provided in assignment 1 shows how to display Python code. Submit the Python code that was used to generate the results as a file called (you can split the code into several .py files; Canvas allows you to submit multiple files). Typing

$ python

should generate all the tables/plots used in your report.


A few general guidelines for this and future assignments in the course:

We will take off points if these guidelines are not followed.

Grading sheet for assignment 5

Part 1:  15 points.

Part 2:  85 points.
( 5 points):  Fixing the code so it handles the bias term correctly
(25 points):  Exploration of a network with a single hidden layer
(25 points):  Exploration of a network with two hidden layers
(15 points):  Regularization
(15 points):  Linear activation function