Skip to content

Level 3 · Advanced In Progress

Goal: assemble Level 1–2 tools into the math that actually trains neural networks — backpropagation derived from the multivariable chain rule, the derivatives behind every common loss function, momentum/Adam as refinements of vanilla gradient descent, regularization, maximum likelihood estimation as the principle behind most loss functions, softmax/cross-entropy, computational graphs and autodiff, batch normalization, and the numerical stability tricks production ML code relies on.

Modules

  1. Backpropagation from the Chain Rule
  2. Loss Functions & Their Derivatives
  3. Optimization Beyond Vanilla GD
  4. Regularization Math (L1/L2)
  5. Maximum Likelihood Estimation
  6. Softmax & Cross-Entropy Derivatives
  7. Computational Graphs & Autodiff
  8. Batch Normalization Math
  9. Numerical Stability in ML Math
  10. Capstone — Hand-Derive a Small Network

By the end of this level you'll be able to hand-derive backpropagation through a small network, derive gradients for MSE/cross-entropy/softmax, explain why Adam converges faster than vanilla gradient descent, and reason about the numerical pitfalls (overflow, underflow, vanishing/exploding gradients) that separate textbook math from working code — with every derivation checked against NumPy/PyTorch-style autodiff.