Topics: How an infinite model can learn from a finite sample. The most important theoretical result in machine learning. Lecture 6 of 18 of Caltech's Machine Learning Course - CS 156 by Professor Yaser Abu-Mostafa. This lecture was recorded on April 19, 2012, in Hameetman Auditorium at Caltech, Pasadena, CA, USA.

1. The Learning Problem (Analysis)
2. The Learning Problem (Analysis)
3. Is Learning Feasible? (Theory)
4. Is Learning Feasible? (Theory)
5. The Linear Model (Technique)
6. The Linear Model (Technique)
7. Error Measures and Noise (Analysis)
8. Error Measures and Noise (Analysis)
9. Training versus Testing (Theory)
10. Training versus Testing (Theory)
11. Theory of Generalization
12. Theory of Generalization
13. The VC Dimension (Theory)
14. The VC Dimension (Theory)
15. Bias-Variance Tradeoff (Theory)
16. Bias-Variance Tradeoff (Theory)
17. The Linear Model II (Technique)
18. The Linear Model II (Technique)
19. Neural Networks (Technique)
20. Neural Networks (Technique)
21. Overfitting (Analysis)
22. Overfitting (Analysis)
23. Technique: Regularization
24. Technique: Regularization
25. Technique: Validation
26. Technique: Validation
27. Support Vector Machines (Theory and Technique)
28. Support Vector Machines (Theory and Technique)
29. Kernel Methods (Theory and Technique)
30. Kernel Methods (Theory and Technique)
31. Radial Basis Functions (Technique)
32. Radial Basis Functions (Technique)
33. Three Learning Principles (Analysis)
34. Three Learning Principles (Analysis)
35. Epilogue: The Map of Machine Learning (Analysis)
36. Epilogue: The Map of Machine Learning (Analysis)

This is an introductory course by Caltech Professor Yaser Abu-Mostafa on machine learning that covers the basic theory, algorithms, and applications. Machine learning (ML) enables computational systems to adaptively improve their performance with experience accumulated from the observed data. ML techniques are widely applied in engineering, science, finance, and commerce to build systems for which we do not have full mathematical specification (and that covers a lot of systems). The course balances theory and practice, and covers the mathematical as well as the heuristic aspects. The course has 18 video lectures, each mainly focused on mathematical (theory), practical (technique), or conceptual (analysis) aspects. Check out the official course website for more information, including a very informative list of topics: https://work.caltech.edu/library/