4
Classification I — Logistic Regression
Summary
From predicting numbers to predicting categories: the sigmoid function as a probability gate, binary cross-entropy loss, gradient descent for classification, decision boundary visualisation, one-vs-all multiclass classification on the Iris dataset, and the XOR problem — which reveals the fundamental limits of linear classifiers and motivates neural networks.
Materials
THEORY
Logistic Regression — Drawing Lines Between Classes
The sigmoid function, decision boundaries, binary cross-entropy loss, gradient descent for classification, one-vs-all multiclass, and why linear classifiers break on XOR.
PRACTICE
Logistic Regression from Scratch in PyTorch
Implement sigmoid, build a custom LogisticRegression layer, code BCE loss, visualise decision boundaries on Iris, extend to one-vs-all multiclass, and watch logistic regression fail on XOR.
Includes notebook