Software

Python modules for socket-based test framework for multiagent reinforcement learning. Centralized environment simulator receives actions from all agents, simulates each step, and returns next states. The framework includes MARL algorithms such as GIGA-WoLF, WPL, Exp3, APQ, and SARSA with both Q-table and function approximation (neural network). In addition to the algorithms, there are built-in games: Matching Pennies, Rock-Paper-Scissors, Traveler's Dilemma, and Prey-vs-Predator.

General python tool set for machine learning and reinforcement learnig. The package includes simple classification and regression modules such as LDA, QDA, linear/nonlinear logistic regression, linear least square, nonlinear regression with neural network, and support vector machines. SVM implements SMO optimization and AOSVR, online optimization for regression. For the learning modules, some utility optimization such as gradient descent and scaled conjugate gradient are included. Reinforcement learning framework is provided with agent model for easy application to various problems. For high dimensional problems, dimensionality reduction methods such as SVD, MNF, CCA, SOM, LLE, and Isomap are included.

Python modules for Extensive Markov Model. This version is translated version of rEMM package. Here are links to the rEMM and EMM.

Octopus arm problem is for learning to control the simulated octopus arm through viscous environment to achieve a certain goal. By controlling each muscle in individual compartment, the agent moves the tip of the arm close to the goal. This is very interesting problem because of its high-dimensional actions, non-linear dynamics, and large search space.

Prototype program for predicting hurricane locations based on Global Forecast System (GFS) analysis. Instead of selecting features manually, the proposed approach train classifiers directly with all the featuture in GFS.