Student Project Ideas
This is a list of projects that I would love to work on with a student.
For most of these topics we can define objectives for a Masters thesis
or a Ph.D. dissertation.
- Reinforcement Learning
-
Reinforcement Learning with Direct Gradient: New RL algorithms
exist for directly learning a policy without first learning a value
function. I have done some preliminary comparisons on simple tasks
and now want to
continue the comparative studies on harder tasks, such as learning to
balance a pole.
-
Reinforcement Learning and Robust Control: By combining these, we
have proved that a system will remain stable while the controller is adapting
via reinforcement learning. We are now testing the resulting algorithm
for the control of a real heating system.
-
Practical Reinforcement Learning by Fast Function Approximation: The
biggest problem with reinforcement learning is that the learning of continuous
functions takes much experience. We are looking at ways of doing this with
less experience.
-
Learning to Walk: Accurate simulations of biped walking robots exist.
I want to apply reinforcement learning algorithms to such a simulation
and watch it learn to walk.
-
Hierarchical Reinforcement Learning : Another way to make reinforcement
learning more practical is to apply it at multiple levels of resolution
over a state space.
-
Expectation-Maximization for Reinforcement Learning : A new way
to train neural networks with fewer parameters than other algorithms.
- Modeling with Neural Networks
-
Modeling How People Learn Sensory-Motor Tasks: A modular neural
network automatically decomposes a modeling problem into small pieces.
When modeling a person's responses who is learning to do a sensory-motor
tasks, these pieces correspond to simple skills that the person has learned.
-
Models of Atmospheric Data: The CSU atmospheric science
department has tons of data exemplifying the relationship between the
parameters of atmospheric models and actual observations. Neural nets
can be used to learn the inverse relationship to directly translate
observations to model parameters.
- Neural Network Implementations
-
Neural Network in SA-C: The Cameron project at CSU
has developed a way to compile C-like programs to run on FPGA's. Much
of their work has focussed on applications in image processing. Since
neural networks involve matrix calculations very much like image
processing tools, it will be relatively easy to see how easily neural
networks can be implemented in SA-C.
-
Neural Network using MMX: The Intel MMX libraries can directly
be used to implement neural network algorithms. The development of a
neural network library based on the MMX libraries would be very
useful. Others have created similar libraries, so comparative studies
would be fairly easy.
- Graphics
-
Volume Registration: An important problem in medical imaging is
the registration of volume data from different scan techniques, like MRI
and PET. I believe neural networks can be used to speed this process.
-
Region Segmentation: Accurate 3-D models of human anatomy can be
developed by painstakingly tracing the borders of regions in 2-D cross-sections.
We have developed a neural net approach that automates most of this. We
need to develop the technique further and find new applications.
-
NURBS from Polygon Meshes: Many CAD tools can manipulate NURBS (non-uniform
rational B-splines), but many models exist only as polygon meshes. Neural
nets can be used to reduce the computation needed to do this.
- EEG
-
Spatial Analysis of EEG: We have developed ways of determining
which mental task a person is doing by analyzing their EEG. To
increase the accuracy, we just study many ways of representing the
signals.
-
New EEG System for BCI: I have constructed an
inexpensive EEG system based on an EEG amplifier and a laptop running
Linux. Now we must test it and perform some preliminary experiments on
using the system as a Brain-Computer Interface.
- Neural Modeling
-
NEURON: Use the NEURON modeling package to simulate field
effects and their role in neuron synchronization. This is in
collaboration with the CSU anatomy department.
Interested? See me, or e-mail to anderson@cs.colostate.edu.