Mohsin Furkh Dar Assistant Professor · UPES
/
Open course

Machine Learning Foundations

Mathematics & ML26 lecturesSelf-pacedNo prerequisites beyond school algebra

Machine learning from first principles, deriving every step rather than quoting it: the functions, calculus, linear algebra and probability it rests on, then the classical algorithms built on top. Interactive lectures with editable Python that runs in the browser.

Start Phase A Phase B Reading

How it works

Every lecture runs in the browser and needs nothing installed. Derivations are hidden one step at a time — predict the next line before you reveal it, and notice when you cannot. Each Python block is editable and executes in the page, so you can change a number, make it fail, and work out why. Exercises come before their solutions on purpose.

The order is the content: each lecture assumes the ones before it. Phase B uses Phase A constantly and does not re-derive it.

Phase A — Mathematics

14 lectures · 5 parts

The functions, calculus, linear algebra and probability that machine learning rests on — built in the order the algorithms need them, and derived rather than quoted.

  • I The Language of Functions
    • 01 What a Function Really Is Every model you will ever train is a function. Before we can talk about learning one, we have to be ruthless about what the word means — and the part almost everyone skips is that a function is not just a formula.
    • 02 Exponentials and Logarithms Machine learning is addicted to logarithms. Log-likelihood, cross-entropy, logits, log-sum-exp, perplexity, KL divergence. This lecture builds the logarithm from its definition so that every one of those is transparent — and so that you never again wonder what log(−2) is.
    • 03 Limits and Continuity A limit answers a question the function itself refuses to: what value is this heading towards at a point where it may not even be defined? Every derivative, every integral, and every convergence guarantee in optimisation is built on this one idea.
  • II Calculus of One Variable
    • 04 The Derivative from First Principles A derivative is a limit of slopes. That single sentence, unpacked carefully, gives you tangent lines, local linear approximation, and the entire mechanism by which a neural network learns.
    • 05 Differentiation Rules and the Chain Rule Four rules replace every difference quotient you would otherwise have to compute. The last of them — the chain rule — is not merely convenient: it is the entire mathematical content of backpropagation, and therefore of how every neural network ever trained was trained.
    • 06 Taylor Series and Local Approximation The tangent line is the best straight-line approximation to a curve. Taylor's theorem says what the best parabola is, the best cubic, and exactly how wrong each of them is — which is where second-order optimisers, learning-rate bounds, and half of numerical analysis come from.
    • 07 Optimisation in One Dimension Training a model is minimising a function. This lecture assembles the complete one-dimensional theory — critical points, the second-derivative test, and convexity — so that when we go to a million dimensions in Lecture 12, only the bookkeeping changes.
  • III Linear Algebra
    • 08 Vectors, Span, and Geometry A data point is a vector. A model's weights are a vector. A gradient is a vector. This lecture builds the object and its geometry — length, angle, projection — because every later result in the course is a geometric statement wearing algebraic clothes.
    • 09 Matrices as Linear Maps A matrix is not a grid of numbers that you multiply by a strange rule. It is a linear function, written down. Once you see that, matrix multiplication stops being arbitrary — it is composition, and the shape rules are just Lecture 1's requirement that the codomain of one map match the domain of the next.
    • 10 Least Squares and the Normal Equations Real data gives you more equations than unknowns, and no exact solution. Least squares is the principled answer: project the target onto what your model can actually reach. This is linear regression, derived twice — geometrically and by calculus — and shown to be the same theorem.
    • 11 Eigenvalues, Eigenvectors, and Quadratic Forms Some directions survive a linear map unchanged except for scaling. Finding them turns a matrix into a list of numbers, turns curvature into geometry, and explains — precisely — why gradient descent zig-zags, why learning rates have a ceiling, and what PCA is doing.
  • IV Multivariable Calculus
    • 12 Gradients, Jacobians, and Hessians Calculus in one variable, plus linear algebra, equals the calculus that machine learning actually runs on. The gradient points uphill, the Jacobian is a derivative that is a matrix, the Hessian is curvature — and the multivariable chain rule is backpropagation.
  • V Probability and Statistics
    • 13 Random Variables, Expectation, and Variance Every loss you minimise is an expectation you cannot compute, estimated from a sample. Understanding what that sentence means — and how wrong the estimate can be — is the difference between running experiments and understanding them.
    • 14 Bayes, Distributions, and Maximum Likelihood The last lecture of Phase A, and the one that ties it together. Maximum likelihood is a single principle from which squared error, cross-entropy, ridge and LASSO all fall out — and Bayes' theorem is the reason a 99%-accurate test can be wrong most of the time.

Phase B — Classical Machine Learning

12 lectures · 6 parts

The algorithms themselves, each one derived from the mathematics of Phase A: what it optimises, what it assumes, and what breaks when the assumption fails.

  • I The Learning Problem
    • 01 What Learning Actually Means Before any algorithm: a precise statement of the problem all of them are trying to solve. Risk, empirical risk, hypothesis classes, and the single rule that separates real machine learning from fooling yourself.
  • II Linear Models
    • 02 Linear Regression, Properly Phase A gave you the normal equations. This lecture turns them into a model you can actually use — basis functions, feature design, coefficient interpretation, residual diagnostics — and is honest about the four ways a linear fit lies to you.
    • 03 Gradient Descent That Actually Works Every model from here on is fitted by gradient descent, so it is worth doing properly. Convergence rates from the condition number, why stochastic gradients have a noise floor, and where momentum and Adam come from — each derived, not quoted.
    • 04 Where Loss Functions Come From You do not choose a loss function. You choose a noise model, and the loss is whatever its negative log-likelihood turns out to be. One principle generates squared error, absolute error, cross-entropy and Poisson loss — and tells you exactly when each is the wrong choice.
    • 05 Logistic Regression and Softmax The workhorse classifier, and the one whose mathematics you must own completely — because the output layer of every classification network is exactly this, and its pathologies are the ones you will meet in every training run.
    • 06 Regularisation: Ridge, LASSO, and Priors We have reached for a penalty term three times already. Here is the proper treatment: every regulariser is a prior, ridge shrinks eigen-directions by a computable factor, and the corners of the L1 ball are precisely why LASSO produces zeros.
  • III Evaluation
    • 07 Bias, Variance, and Honest Evaluation Why every model's error splits into three parts, why cross-validation is the right tool for choosing between them, and which of the metrics everyone reports are actively misleading.
  • IV Unsupervised Learning
    • 08 Principal Component Analysis Three completely different questions — which direction has most variance, which subspace loses least information, and what does the SVD of the data matrix say — all have the same answer. That coincidence is PCA, and it is the most useful thing the spectral theorem does.
    • 09 k-means, Gaussian Mixtures, and EM Clustering without labels. k-means is alternating minimisation of a squared-error objective; Gaussian mixtures are its probabilistic upgrade; and EM — the algorithm that fits them — is one of the most reused ideas in the subject.
  • V Generative Models
    • 10 Naive Bayes and the Generative–Discriminative Divide Instead of modelling p(y|x) directly, model how the data was made and invert with Bayes' theorem. The 'naive' assumption is almost always false, the classifier is often excellent anyway, and understanding why teaches you more about learning than the algorithm itself.
  • VI Nonlinear Methods
    • 11 Decision Trees, Random Forests, and Gradient Boosting Leave the hyperplane behind. A tree carves the input space into boxes; a forest averages many de-correlated trees; boosting fits each new tree to what the ensemble still gets wrong. Two of these are still the best thing to run on tabular data.
    • 12 Support Vector Machines: Margins, Duality, and Kernels Keep the linear boundary; change the space it lives in. Getting there requires the maximum-margin principle, Lagrangian duality, and the observation that the dual touches the data only through inner products — which is the whole kernel trick in one sentence.

Phases C & D

Planned

Listed so the mathematics in Phase A has a visible destination. Neither phase is written yet.

  • C Statistical Learning Theory & Deep Learning Generalisation and empirical risk minimisation, then computational graphs, automatic differentiation, backpropagation, convolutional and recurrent networks, attention and Transformers. Not written yet
  • D Advanced & Research Machine Learning Probabilistic graphical models, variational inference, Monte Carlo methods, generative models, reinforcement learning and information theory. Not written yet

Reading

22 books

Nothing here is hosted on this site. Each link goes to the author’s or publisher’s own copy. The first group is given away free by the people who wrote it.

Free from the author or publisher

In print