Differences

This shows you the differences between two versions of the page.

--- start [2022/09/07 10:56] – [September] anderson
+++ start [2024/09/19 10:43] (current) – [September] anderson
@@ Line 6: / Line 6: @@
 ***/
+/***
+Please send your suggestions regarding lecture topics to Chuck using [[https://tinyurl.com/2nyfzc36|this Google Docs form]].  Questions regarding assignments should be entered in Canvas discussions.
+***/
+ \\
+ \\
+ \\
 The following schedule is **tentative and is being updated**.
-Please send your suggestions regarding lecture topics to Chuck using [[https://tinyurl.com/2nyfzc36|this Google Docs form]].  Questions regarding assignments should be entered in Canvas discussions.
+All students may attend the lecture remotely using [[https://zoom.us/j/92107238733?pwd=Wggv0JQGepdeoezMRrv0gpVImn90yl.1|this zoom link]].
 ===== August =====
 |< 100% 18% 20% 22% 20% 20%  >|
-^  Week      ^  Topic      ^  Material  ^  Reading          ^  Assignments  ^
+^  Week      ^  Topic      ^  Lecture Notes  ^  Reading          ^  Assignments  ^
-| Week 1:\\  Aug 23, 256   | Overview of course. Review of neural networks training and use.  | [[http://nbviewer.ipython.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/01 Introduction to CS545.ipynb|01 Introduction to CS545]]\\ [[http://nbviewer.ipython.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/02 Searching for Good Weights in a Linear Model.ipynb|02 Searching for Good Weights in a Linear Model]]   | [[https://jupyterlab.readthedocs.io/en/stable/getting_started/overview.html|JupyterLab Introduction]], watch the video then play with jupyter lab.  \\ [[https://tinyurl.com/2qw45tlp|The Batch]] from DeepLearning.AI. Yay, Colorado!   |   |
+| Week 1:\\  Aug 20, 22   | Course overview.  \\ Machine Learning and AI: History and Present Boom\\ Jupyter notebooks.    | [[https://nbviewer.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/01 Introduction to CS545.ipynb|01 Introduction to CS545]]\\ [[https://nbviewer.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/01a Simple Animations.ipynb|01a Simple Animations]]\\  [[https://nbviewer.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/02 Searching for Good Weights in a Linear Model.ipynb|02 Searching for Good Weights in a Linear Model]]    | [[https://jupyterlab.readthedocs.io/en/stable/getting_started/overview.html|JupyterLab Introduction]], watch the video then play with jupyter lab.  \\ [[https://www.freecodecamp.org/news/exploratory-data-analysis-with-numpy-pandas-matplotlib-seaborn/|What is Data Analysis? How to Visualize Data with Python, Numpy, Pandas, Matplotlib & Seaborn Tutorial]], by Aakash NS| Not graded: Please fill out [[https://forms.gle/hppJ5QuRFuRn1L2h7|this anonymous survey]] before Thursday class.  |
-| Week 2:\\  Aug 30, Sept 1  | Thursday lecture cancelled. Please watch pre-recorded lecture in Echo360. Quiz1 and A1 questions. Regression with neural networks.  | [[http://nbviewer.ipython.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/03 Fitting Simple Models Using Gradient Descent in the Squared Error.ipynb|03 Fitting Simple Models Using Gradient Descent in the Squared Error]]  |  |[[https://nbviewer.ipython.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/Quiz1.ipynb|Quiz 1]] due Wednesday, August 31, 10:00 PM, in Canvas  |
+| Week 2:\\  Aug 27, 29  | Optimization algorithms. Simple linear and nonlinear models.  Confidence intervals.   | [[https://nbviewer.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/02 Searching for Good Weights in a Linear Model.ipynb|02 Searching for Good Weights in a Linear Model]] \\  [[https://nbviewer.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/02a Input Importance and Generative AI---Friend or Foe.ipynb|02a Input Importance and Generative AI---Friend or Foe]] \\ [[https://nbviewer.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/03 Fitting Simple Models Using Gradient Descent in the Squared Error.ipynb|03 Fitting Simple Models Using Gradient Descent in the Squared Error]]\\ [[https://nbviewer.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/04 Training Multiple Models to Obtain Confidence Intervals.ipynb|04 Training Multiple Models to Obtain Confidence Intervals]]   |   |  |
 ===== September =====
 |< 100% 18% 20% 22% 20% 20%  >|
-^  Week      ^  Topic      ^  Material  ^  Reading          ^  Assignments  ^
+^  Week      ^  Topic      ^  Lecture Notes  ^  Reading          ^  Assignments  ^
-| Week 3:\\  Sept 6, 8  | Optimizers. Neural Network class.  | [[http://nbviewer.ipython.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/04 Introduction to Neural Networks.ipynb|04 Introduction to Neural Networks]]\\ [[http://nbviewer.ipython.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/05 Optimizers.ipynb|05 Optimizers]]  | [[https://doi.org/10.1016/j.neucom.2022.06.111|Activation functions in deep learning: A comprehensive survey and benchmark]], Neurocomputing, volume 503, 2022, pp. 92-108  |   |
+| Week 3:\\  Sept 3, 5  | Introduction to neural networks.   | [[https://nbviewer.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/05 Introduction to Neural Networks.ipynb|05 Introduction to Neural Networks]]  | [[https://www.3blue1brown.com/topics/neural-networks|3Blue1Brown Introduction to Neural Networks]] in the first five chapters provides a fun video tutorial including error backpropagation.  |   |
-| Week 4:\\  Sept 13, 15  | A2. Autoencoders. Classification.   | [[http://nbviewer.ipython.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/06 Autoencoders.ipynb|06 Autoencoders]]\\ [[http://nbviewer.ipython.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/07 Introduction to Classification.ipynb|07 Introduction to Classification]]  | | [[http://nbviewer.ipython.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/A1 Three-Layer Neural Network.ipynb|A1 Three-Layer Neural Network]] due Monday, Sept 12th, at 10:00 PM   |
+| Week 4:\\  Sept 10, 12   | Design of NeuralNetwork class. Optimizers. Overview of A2. Memory organization for neural network parameters. Optimizers tailored for neural networks.  | [[https://nbviewer.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/06 Python Classes.ipynb|06 Python Classes]]\\ [[https://nbviewer.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/07 Optimizers Simple.ipynb|07 Optimizers Simple]]\\ [[https://nbviewer.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/08 Collecting All Weights into One-Dimensional Vector for Use in Optimizers.ipynb|08 Collecting All Weights into One-Dimensional Vector for Use in Optimizers]]\\ [[https://nbviewer.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/08a Optimizers.ipynb|08a Optimizers]]   | [[https://machinelearningmastery.com/weight-initialization-for-deep-learning-neural-networks/|Weight Initialization for Deep Learning Neural Networks]], by Jason Brownlee  | [[https://nbviewer.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/A1.ipynb|A1]] due Monday, September 9th, 10:00 PM.  |
-| Week 5:\\  Sept 20, 22  | Classification.  | [[http://nbviewer.ipython.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/08 Classification with Linear Logistic Regression.ipynb|08 Classification with Linear Logistic Regression]]\\ [[http://nbviewer.ipython.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/09 Classification with Nonlinear Logistic Regression Using Neural Networks.ipynb|09 Classification with Nonlinear Logistic Regression Using Neural Networks]]  |
+| Week 5:\\  Sept 17, 19\\ Chuck's office hours cancelled today.  | Introduction to Classification.  | [[https://nbviewer.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/09 Introduction to Classification.ipynb|09 Introduction to Classification]]    | | [[https://nbviewer.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/A2 NeuralNetwork Class.ipynb|A2 NeuralNetwork Class]] due <color red>Thursday, September 19, 10:00 PM.</color> Notebook and A2grader updated Sept. 12, 5:30 pm.   |
-| Week 6:\\  Sept 27, 29  |   | [[http://nbviewer.ipython.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/10 JAX.ipynb|10 JAX]]\\ [[https://www.cs.colostate.edu/~anderson/cs545/notebooks/neuralnetworks_app.tar|neuralnetworks_app.tar]]  | [[https://moocaholic.medium.com/jax-a13e83f49897|JAX Ecosystem]]\\ [[https://streamlit.io/|Streamlit]]  | |
+| Week 6:\\  Sept 24, 26  | Early stopping (new version of optimizers). A3. Introduction to classification.   |   |
 ===== October =====
 |< 100% 18% 20% 22% 20% 20%  >|
-^  Week      ^  Topic      ^  Material  ^  Reading          ^  Assignments  ^
+^  Week      ^  Topic      ^  Lecture Notes  ^  Reading          ^  Assignments  ^
-| Week 7:\\  Oct 4, 6  | Convolutional neural networks.  | [[http://nbviewer.ipython.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/11 Convolutional Neural Networks.ipynb|11 Convolutional Neural Networks]]\\ [[https://www.cs.colostate.edu/~anderson/cs545/notebooks/CNN Backprop.pdf|CNN Backpropagation Notes]]  | [[https://spectrum.ieee.org/special-reports/the-great-ai-reckoning/|The Great AI Reckoning]]
+| Week 7:\\  Oct 1, 3  | Classification with QDA, LDA, and linear logistic regression.  |  | |  |
-| Week 8:\\  Oct 11, 13  | Pytorch. Convolutional neural nets  | [[http://nbviewer.ipython.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/12 Introduction to Pytorch.ipynb|12 Introduction to Pytorch]]\\ [[http://nbviewer.ipython.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/13 Convolutional Neural Networks in Pytorch.ipynb|13 Convolutional Neural Networks in Pytorch]]\\ [[http://nbviewer.ipython.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/14 Convolutional Neural Networks in Numpy.ipynb|14 Convolutional Neural Networks in Numpy]]  |
+| Week 8:\\  Oct 8, 10  | Classification with Nonlinear Logistic Regression. Introduction to Reinforcement Learning.  |  | |
-| Week 9:\\  Oct 18, 20  | Reinforcement Learning  | [[http://nbviewer.ipython.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/15 Introduction to Reinforcement Learning.ipynb|15 Introduction to Reinforcement Learning]]\\ [[http://nbviewer.ipython.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/16 Reinforcement Learning with Neural Network as Q Function.ipynb|16 Reinforcement Learning with Neural Network as Q Function]]  | |   |
+| Week 9:\\  Oct 15, 17  | Reinforcement learning with Q Function as Neural Network. Learning to play games. |   | [[https://lastweekin.ai/p/241|Last Week in AI]]\\ [[https://www.cbsnews.com/news/geoffrey-hinton-ai-dangers-60-minutes-transcript/?utm_source=substack&utm_medium=email|Geoffrey Hinton: AI Dangers, on 60 Minutes]]  |    |
-| Week 10:\\  Oct 25, 27  | Reinforcement Learning  | [[http://nbviewer.ipython.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/17 Reinforcement Learning for Two Player Games.ipynb|17 Reinforcement Learning for Two Player Games]]\\ [[http://nbviewer.ipython.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/18 Reinforcement Learning to Control a Marble.ipynb|18 Reinforcement Learning to Control a Marble]]\\ [[http://nbviewer.ipython.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/19 Reinforcement Learning Modular Framework.ipynb|19 Reinforcement Learning Modular Framework]] |
+| Week 10:\\  Oct 22, 24  | Modular framework for reinforcement learning. Convolutional Neural Networks.   |   | |   |
+| Week 11:\\  Oct 29, 31  | Pytorch.\\ Jax.\\ Ray.    |   | [[https://www.whitehouse.gov/briefing-room/statements-releases/2023/10/30/fact-sheet-president-biden-issues-executive-order-on-safe-secure-and-trustworthy-artificial-intelligence/|President Biden's Executive Order on Safe, Secure, and Trustworthy Artificial Intelligence]]  |   |
 ===== November =====
 |< 100% 18% 20% 22% 20% 20%  >|
-^  Week      ^  Topic      ^  Material  ^  Reading          ^  Assignments  ^
+^  Week      ^  Topic      ^  Lecture Notes  ^  Reading          ^  Assignments  ^
-| Week 11:\\  Nov 1, 3  | Transfer learning in Reinforcement Learning.\\ Brain-Computer Interfaces  | Slide presentations  | [[http://www.cs.colostate.edu/~anderson/wp/pubs/pretrainijcnn15.pdf|Faster Reinforcement Learning After Pretraining Deep Networks to Predict State Dynamics]], [[https://ieeexplore.ieee.org/document/9533751|Increased Reinforcement Learning Performance through Transfer of Representation Learned by State Prediction Model]]
+| Week 12:\\  Nov 5, 7  | Convolutional Neural Networks.  |  | |
-| Week 12:\\  Nov 8, 10  | BCI. Recurrent Neural Networks. | [[http://nbviewer.ipython.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/20 Recurrent Networks in Numpy.ipynb|20 Recurrent Networks in Numpy]]\\ [[http://nbviewer.ipython.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/21 Recurrent Networks in Pytorch.ipynb|21 Recurrent Networks in Pytorch]]\\ [[http://nbviewer.ipython.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/22 Classifying EEG Using Recurrent Neural Networks.ipynb|22 Classifying EEG Using Recurrent Neural Networks]]  | |  |
+| Week 13:\\  Nov 12, 14  | Ensembles. Mixture of Experts.   |   |   |
-| Week 13:\\  Nov 15, 17  | K-means clustering. K-nearest-neighbor classification. Support Vector Machines.   | [[http://nbviewer.ipython.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/23 K-Means Clustering, K-Nearest-Neighbor Classification.ipynb|23 K-Means Clustering, K-Nearest-Neighbor Classification]]\\ [[http://nbviewer.ipython.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/24 Support Vector Machines.ipynb|24 Support Vector Machines]]    |
+| Week 14:\\  Nov 19, 21  | Clustering. K-Nearest Neighbors. Web Apps with Streamlit.  |   | [[https://www.nature.com/articles/d41586-023-03635-w|ChatGPT generates fake data set to support scientific hypothesis]]  |   |
-| Week 14:\\  Nov 29, Dec 1  | Introduction to Transformers  | [[http://nbviewer.ipython.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/25 Introduction to Transformers.ipynb|25 Introduction to Transformers]] |
+| Fall Break:\\ Nov 25-29 | No classes.  |
 ===== December =====
 |< 100% 18% 20% 22% 20% 20%  >|
-^  Week      ^  Topic      ^  Material  ^  Reading          ^  Assignments  ^
+^  Week      ^  Topic      ^  Lecture Notes  ^  Reading          ^  Assignments  ^
-| Week 15:\\  Dec 6, 8  | Transformers: Self-Attention Replaced by Fourier Transform.\\ Cascade Ensemble Network   | [[http://nbviewer.ipython.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/26 FNet--Replace Self-Attention with Fourier Transform.ipynb|26 FNet--Replace Self-Attention with Fourier Transform]]\\ [[http://nbviewer.ipython.org/url/www.cs.colostate.edu/~anderson/cs545/notebooks/27 Cascade Ensemble Network.ipynb|27 Cascade Ensemble Network]] |
+| Week 15:\\  Dec 3, 5  | Word embeddings. Transformers.  |   |     | |
-| Dec 12-16  |  Final Exam Week  |  No Exams in this course  |
+| Dec 10-12  |  Final Exam Week  |  No Exams in this course  | |   |