CS540: Topics for Midterm Exam

Neural Networks

• Single unit computations
  • linear (weighted sum)
  • hard threshold; yes/no decisions
  • soft threshold; sigmoid functions
  • Be able to calculate output of sigmoid, given weights and input. You won't need a calculator. Answers may include exp function.
  • Be able to find weight values for unit given at least two desired input-output pairs. Answers may include ln function.
• Multiple layers for more complex computations
  • Why hidden units add complexity.
  • Why XOR problem requires at least two hidden units.
• Training a neural network
  • Know the derivative of the sigmoid
  • Error backpropagation
    • weight update rules for a simple neural network with two output units and one hidden unit.
    • Batch versus incremental updating
    • Over-fitting; What it is. How to minimize by early stopping.
    • Cross-validation
  • Confidence interval formula. Assumes normal distribution.
  • Bagging and Boosting.
  • What are they? How do they differ?
  • Why might you get better results with these than you would with a single network?
• How can you determine with inputs to a neural network are most significant?

Reinforcement Learning

• Definitions
  • policy
  • value, or utility, function
  • action-value function
  • transition probability matrix
  • Markov Decision Problem
  • Partially-Observable Markov Decision Problem
• Know the definition of and how to apply the following procedures:
  • Value Iteration
  • Policy Iteration
  • Q Learning
• Temporal credit assignment problem
• Why are function approximators useful in reinforcement learning; curse of dimensionality

Uncertainty

• Why a useful addition to logical reasoning? laziness and ignorance
• Prior and conditional probabilities
• Probability distribution
• Joint probabilities
• P(A^B) = P(A|B) P(B)
• Multivalued random variables; P(X|Y)
• Know the three axioms
• Bayes' Rule
  • P(B|A) = P(A|B) P(B) / P(A)
  • Calculating the denominator by normalization: expected value of P(A|x) for all x.
  • Bayesian updating with assumption of conditional independence. Know and be able to use.
    • P(A|B^C) = P(A) [ P(B|A) / P(B) ] [ P(C|A) / P(C|B) ]