Colorado State University CS681: Reinforcement Learning and Neural Networks (Spring, 99)

Computer Science Department
Colorado State University

Contents (links to sections on this page)

What's New?

Go back to Contents. 

Schedule

Date Read Through Written Exercises Due Programming Exercises Due
Jan 21 (Th) 1.7 Ex: 1.4
Jan 26 (Tu) 2.6
Jan 28 (Th) 2.12 Ex: 2.2
Feb 2 (Tu) 3.1 Ex: 2.12, 2.13, 2.15, 2.16 Ex: 2.14
Feb 4 (Th) 3.7
Feb 9 (Tu) 3.11 Ex: 3.7-3.13
Feb 11 (Th) 4.3 Ex: 3.14-3.17
Feb 16 (Tu) 4.9 Ex: 4.3, 4.6
Feb 18 (Th) 5.5 Ex: 4.7, 4.9
Feb 23 (Tu) 5.9 Ex: 4.8
Feb 25 (Th) 6.2 Ex: 5.3, 5.6, 5.7
Mar 2 (Tu) No Class
Mar 4 (Th) 6.10 Ex: 6.1-6.5 Ex: 5.4
Mar 16 (Tu) 7.4 Ex: 6.8, 6.9,6.10 Ex: 6.6
Mar 18 (Th) 7.12 Ex: 7.1, 7.2, 7.4
Mar 23 (Tu) 8.3 Ex: 7.6, 7.8 Ex: 7.7
Mar 25 (Th) Neural Nets Ex: 8.1, 8.3, 8.5, 8.7
Mar 30 (Tu) 8.8
Apr 1 (Th) 11.3 SARSA(lambda)-NN on Mountain Car
Apr 6 (Tu) 11.6
Apr 8 (Th) Proposal Presentations Written Proposal
Apr 13 (Tu) More Neural Nets
Apr 15 (Th) 9.3
Apr 20 (Tu) 9.9
Apr 22 (Th) 10.2
Apr 27 (Tu) Papers Sutton, "Generalization in Reinforcement Learning: Successful Examples Using Sparse Coarse Coding"; Samuel, "Some Studies in Machine Learning Using the Game of Checkers"; Zhang and Dietterich, "Solving Combinatorial Optimization Tasks by Reinforcement Learning: A General Methodology Applied to Resource-Constrained Scheduling"; Lang and Waibel, "A Time-Delay Neural Network Architecture for Isolated Word Recognition;
Apr 29 (Th) Papers Sick, "Structure Evolution for Time-Delay Neural Networks; Mataric, "Reinforcement Learning in the Multi-Robot Domain"; Randlov, "Learning Macro-Actions in Reinforcement Learning"
May 4 (Tu) Papers Gadeleta and Danglemayr, "Optimal Chaos Control Through Reinforcement Learning", Anderson, "Q-Learning with Hidden-Unit Restarting"
May 6 (Th) Cancelled
May 8 (Sat) 3 pm - 6 pm, usual classroom Project Presentations
May 8 (Sat) 6 pm - 7 pm, HP classroom Project Demonstrations
May 12 (Wed) Noon Written Projects Due

Go back to Contents. 

Resources

Go back to Contents. 

Course Description

Reinforcement learning refers to a broad class of learning problems and algorithms that share the goal of optimizing the behavior of an agent that can be modified with experience. An agent can be a computer program, a control device in a feedback system, or an autonomous robot. In class we will review the history of reinforcement learning, starting from models of animal learning to recent theoretical developments relating reinforcement learning and dynamic programming methods. Neural networks will be discussed as one way of learning the value and policy functions in dynamic programming and reinforcement learning.

Students are expected to read the assigned chapters and be prepared to discuss them in class. Exercises from the textbook will be assigned. Some of the exercises will be programming assignments; any programming language may be used. Students will also complete and present in class a semester project involving reinforcement learning or neural networks.

Time and Place

Instructor

Text Book

Prerequisites

To register for CS681, you must have taken CS540, or have the instructor's approval.

Grading

Your grade will be based on homework assignments, semester project, and participation in class discussions, weighted as follows: Go back to Contents. 
Copyright © 1999 Charles W. Anderson,
anderson@cs.colostate.edu