When looking for the best learning rates, be brave. Start with high values, like 1 or 10. Look at the training error versus epochs. If it is widely nonmonotonic, then the rate is too high. Cut it in half several times, watching for training error curves that are well behaved. This gives you an upper bound. Then try very small values, like 0.001 or 0.01. See if the training error decreases significantly within the number of epochs you want to run. This gives you a lower bound. From now on you can try a systematic search. Try a set of learning rates between your lower and upper bounds. It usually works well to space them in increasing intervals. Say your lower and upper bounds are 0.001 and 1. You could try values that are approximately 0.001 times multiples of 2, i.e., 0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.1, 0.2, 0.5, 1. If you want to search further, pick the best value tried so far and search around it at smaller intervals. For the momentum rate, always try 0 and 0.9. You can try intermediate values, or higher values between 0.9 and 0.9999, but generally I find it is sufficient to just check 0 and 0.9.
Format your assignment reports as serious papers---something you might submit to a conference. The following points relate to this.
Figures must appear right side up, be a reasonable size, and be in black and white or grayscale with a white background. They must be numbered and have a lengthy caption of several sentences. The captions should be informative enough for a reader to get a rough idea of the figure contents without reading the paper. Figures must always be referred to in your text and explained in detail in the text. Tell reader what the axes units are, what values are plotted, the conclusions you draw from the results in the figure, and so on.
Run a spelling checker.
Include a short abstract that summarizes the objective of your work and the conclusions. Try to entice the reader into reading your paper.
Use section numbers and titles. For this assignment you could use: