State-action Rewards, Finite Horizon MDPs, Dynamical Systems:  Models, LQR, Riccati Equation 
State-action Rewards, Finite Horizon MDPs, Dynamical Systems: Models, LQR, Riccati Equation by Stanford / Andrew Y. Ng
Video Lecture 18 of 20
1 rating
Views: 4,167
Date Added: September 13, 2008

Lecture Description

Lecture by Professor Andrew Ng for Machine Learning (CS 229) in the Stanford Computer Science department. Professor Ng discusses state action rewards, linear dynamical systems in the context of linear quadratic regulation, models, and the Riccati equation, and finite horizon MDPs.

Course Index

  1. Introduction: Applications of Machine Learning, Logistics of the Class, Learning Theory
  2. Linear Regression, Gradient Descent, Normal Equations
  3. Locally Weighted Regression, Probabilistic Interpretation, Logistic Regression
  4. Newton's Method, Exponential Family, Generalized Linear Models (GLMs)
  5. Learning Algorithms: Generative, Gaussian Discriminant Analysis, Digression
  6. Neural networks, Naive Bayes, Support Vector Machines
  7. Optimal Margin Classifier, Primal/Dual Optimization (KKT), SVM Dual, Kernels
  8. Support Vector Machines: Kernels, Soft Margin, SMO Algorithm
  9. Learning Theory I: Bias/Variance, ERM, Union Bound, Hoeffding Inequality, Uniform Convergence
  10. Learning Theory II: Uniform Convergence, VC Dimension, Model Selection
  11. Bayesian Statistics and Regularization, Online Learning, Machine Learning Algorithms
  12. Unsupervised Learning, Clustering, Mixtures of Gaussians, Jensen's Inequality, EM Algorithm
  13. Expectation Maximization: Mixture of Gaussians, Naive Bayes, Factor Analysis
  14. The Factor Analysis EM steps, Principal Component Analysis (PCA)
  15. PCA: Latent Semantic Indexing, Singular Value Decomposition, Independent Component Analysis
  16. Reinforcement Learning I: MDP, Value Function, Value Interation, Policy Iteration
  17. Reinforcement Learning II: Continuous State MDP, Discretization, Models/Simulators, Fitted Value
  18. State-action Rewards, Finite Horizon MDPs, Dynamical Systems: Models, LQR, Riccati Equation
  19. Debugging RL Algorithm, Linear Quadratic Regulation, Kalmer Filters, Linear Quadratic Gaussian
  20. POMDPs, Pegasus Algorithm, Pegasus Policy Search, Applications of Reinforcement Learning

Course Description

This course (CS229) -- taught by Professor Andrew Ng -- provides a broad introduction to machine learning and statistical pattern recognition. Topics include supervised learning, unsupervised learning, learning theory, reinforcement learning and adaptive control. Recent applications of machine learning, such as to robotic control, data mining, autonomous navigation, bioinformatics, speech recognition, and text and web data processing are also discussed.


There are no comments. Be the first to post one.
  Post comment as a guest user.
Click to login or register:
Your name:
Your email:
(will not appear)
Your comment:
(max. 1000 characters)
Are you human? (Sorry)