ML concepts, explained so they stick

How to learn ML How to learn machine learning in 2026: a complete roadmap for self-taught learners The order to learn machine learning on your own: Python, the math that ML uses, PyTorch, then training and evaluating deep networks, with every concept named so you know exactly what is ahead of you. 15 September 2026 · 14 min read How to learn ML How to learn machine learning effectively (and actually remember it) Most people finish an ML course and cannot use it a month later. This post covers what to learn in which order, how to practice so the concepts stay, and a weekly schedule you can keep with 30 to 45 minutes a day. 16 September 2026 · 7 min read What NLP is How to learn NLP in 2026: a complete roadmap from text to LLMs What to learn for natural language processing and in what order, from tokenization and word embeddings to transformers, LLMs and post-training, so you can see what is ahead and check off what you already know. 13 October 2026 · 14 min read

Word representations Bag of words, co-occurrence matrices, and PMI: count-based word vectors How counting the words around each word turns raw text into vectors, with PyTorch code for a bag-of-words vector, a co-occurrence matrix and PPMI, and what each of them loses. 30 October 2026 · 8 min read Word representations How to turn words into numbers: integer encoding, one-hot, and dense vectors How to measure whether two word vectors are close with distance and cosine similarity, why token ids and one-hot vectors cannot show which words are related, and what a dense vector changes. 30 October 2026 · 12 min read Word representations What are word embeddings? How an embedding table turns a token id into a vector in PyTorch, why training can move those vectors, and how relationships like king to queen show up as directions, with the bug that breaks analogy search. 30 October 2026 · 5 min read Tokenization BPE vs WordPiece vs Unigram tokenization A worked example where BPE and WordPiece merge different pairs, how the ## marker works, and the score Unigram uses to choose between ways of cutting a word. 29 October 2026 · 7 min read Tokenization How to use a Hugging Face tokenizer: special tokens, token IDs, padding, and batches How to load a pretrained tokenizer and read what it returns, the four stages a string passes through, and how to batch sentences of different lengths with padding and truncation. 29 October 2026 · 11 min read Tokenization Why LLMs miscount letters: tokenization quirks and non-English text What GPT-2's tokenizer does to a word, a number and an emoji, how many tokens one sentence takes in four languages, and why some tokens make a model behave strangely. 29 October 2026 · 6 min read Tokenization Byte pair encoding (BPE) explained step by step, including byte-level BPE How BPE builds a subword vocabulary from counts alone, a worked example you can check by hand, the three short Python functions behind it, and why byte-level BPE never needs an unknown token. 28 October 2026 · 10 min read Text as data Privacy and licensing: can you train a model on scraped text? What to check before you collect or publish a text dataset, what a dataset license spells out, and a short Python example that masks emails, phone numbers and IP addresses. 28 October 2026 · 5 min read Tokenization What is tokenization in NLP? Word, character, and subword tokens Why a language model needs text cut into pieces first, the three questions that decide how big a piece should be, and why most models today use subword tokens instead of whole words or single characters. 28 October 2026 · 16 min read Text as data How to handle class imbalance in text classification Why a spam filter can score 98% accuracy while catching no spam, which numbers to report instead, and how reweighting, oversampling and undersampling work in PyTorch, with what each one costs. 26 October 2026 · 6 min read Text as data Data quality problems in NLP: duplicates, contamination, bias, and representativeness The problems to check for in a text dataset before training, how each one changes a model's scores, and short Python checks for duplicates, train/test overlap and test-set size. 26 October 2026 · 10 min read Text as data Text preprocessing in NLP: normalization, stemming vs lemmatization, and stopwords What each classic cleanup step does to a sentence, with runnable NLTK code, where each one erases something a model could use, and how much of it to apply for a count-based model versus a transformer. 26 October 2026 · 9 min read Text as data Data annotation for NLP: crowdsourcing vs experts, and inter-annotator agreement How to design labels people can apply consistently, when to pay a crowd and when to hire experts, and how to compute Cohen's kappa to check whether your labels can be trusted. 24 October 2026 · 8 min read Text as data How to build your own NLP dataset: selecting data, extracting text, documenting it How to choose the text for a custom dataset and pull clean text out of web pages, PDFs and scans, plus what to write down so the next person can trust it. 24 October 2026 · 7 min read Text as data Where to find NLP datasets: Hugging Face Hub, Kaggle, and more Which places to search for a text dataset, how to load one in Python, and what to check before you train anything on it. 24 October 2026 · 7 min read What NLP is A short history of NLP: from ELIZA to deep learning The four eras NLP went through on its way to today's large models, why each shift happened, and what the 1960s chatbot ELIZA shows about sounding smart without understanding anything. 23 October 2026 · 7 min read Text as data Unicode and UTF-8 explained: how computers store text How a character becomes a code point and then bytes, why the same word can take a different number of bytes in different scripts, and how to fix text that compares wrong or shows up garbled. 23 October 2026 · 6 min read Text as data Why text is hard data for machine learning How raw text differs from tables and images, and the three decisions an NLP system has to make before a model can learn anything from it. 23 October 2026 · 7 min read Numerical computation Exponential moving average (EMA) explained How the EMA update works, why it starts too low and how bias correction fixes that, and how to read a beta like 0.9 or 0.999 as a window. 22 October 2026 · 4 min read Numerical computation The log-sum-exp trick explained: overflow, underflow, and adding logs Why exp(1000) returns inf, why a product of 500 small probabilities returns 0, and how factoring out the largest value lets you add numbers stored as logs in PyTorch. 22 October 2026 · 5 min read What NLP is What is NLP? Natural language processing explained The main NLP tasks from sentiment analysis to language modeling, the kinds of ambiguity that make human language hard for a computer, and why modern NLP learns from data instead of following hand-written rules. 22 October 2026 · 9 min read Numerical computation Floating-point numbers explained: float32, float64, and rounding errors Why 0.1 + 0.2 is not 0.3, how float32 and float64 differ, when a small number vanishes or a subtraction goes wrong, and how to compare floats safely in Python and PyTorch. 21 October 2026 · 5 min read Information theory Mutual information explained How to compute mutual information from a joint probability table by hand and in PyTorch, how it connects to entropy and KL divergence, and where it is used in feature selection and decision trees. 21 October 2026 · 7 min read Information theory What is entropy in information theory? How to compute entropy by hand and in PyTorch, what bits and nats are, why it can go negative for continuous variables and how the maximum entropy principle picks a distribution from a few known facts. 21 October 2026 · 9 min read Probability and statistics Bootstrap resampling explained How to get a standard error and a 95% interval for a median, an accuracy or any other statistic without deriving a formula, with the PyTorch code and the mistakes to avoid. 20 October 2026 · 5 min read Probability and statistics Markov chain Monte Carlo (MCMC) explained How the Metropolis-Hastings rule decides each step, why the unknown normalizing constant never has to be computed, and how many independent draws a correlated chain is worth, with a working sampler in PyTorch. 20 October 2026 · 6 min read Probability and statistics The reparameterization trick explained How to build a Gaussian sample from a mean, a Cholesky factor and standard normal noise, why that lets gradients reach the mean, and how PyTorch's rsample does it for you. 20 October 2026 · 5 min read Probability and statistics Bayesian inference explained: priors, posteriors, and conjugate priors How to turn a prior and a likelihood into a posterior, and how to read uncertainty and predictions off it. The PyTorch examples also update a Beta prior by counting. 19 October 2026 · 7 min read Probability and statistics Bias and variance of an estimator, and why sample variance divides by n minus 1 How to measure an estimator's bias and variance with a simulation, how they add up to mean squared error, and how to choose between the two sample variance formulas in PyTorch. 19 October 2026 · 6 min read Probability and statistics Monte Carlo methods explained: Monte Carlo estimation, importance sampling, and rejection sampling What error to expect from an average of random samples, and how importance sampling and rejection sampling handle targets that are hard to sample from. The PyTorch examples also show why both struggle in high dimensions. 19 October 2026 · 7 min read Probability and statistics Hypothesis testing and p-values: is model A really better than model B? How to tell whether a gap between two models is real or noise, what a p-value does and does not mean, one-sided vs two-sided tests, and why running many tests produces false winners, with PyTorch checks. 18 October 2026 · 8 min read Probability and statistics MLE vs MAP estimation How maximum likelihood and maximum a posteriori estimation each pick a parameter from data, why MAP gives a sensible answer where MLE overreacts, and how a prior turns into L2 or L1 regularization, with PyTorch examples. 18 October 2026 · 8 min read Probability and statistics Standard error and confidence intervals explained How to put an error bar on an average, such as a validation accuracy or a mean latency, how to compute it by hand and in PyTorch, and what 95% confidence does and does not promise. 18 October 2026 · 7 min read Probability and statistics CDF, percentiles, and quantiles explained How to compute a CDF for discrete and continuous data, how to read a median or a 95th percentile off it, and how to do both in PyTorch on real latency data. 17 October 2026 · 7 min read Probability and statistics Covariance vs correlation How to compute covariance and correlation by hand and in PyTorch, why correlation is the one to compare across datasets, and how covariance decides whether the uncertainty of two quantities adds up or cancels. 17 October 2026 · 6 min read Probability and statistics Expected value and variance explained, with conditional expectation How to compute the expected value and variance of a distribution by hand and in PyTorch, what happens to them when you shift or scale the data, and why a model trained with squared error predicts a conditional mean. 17 October 2026 · 8 min read Probability and statistics Kernel density estimation explained How KDE turns a handful of data points into a smooth density by hand and in PyTorch, how the bandwidth h changes the result, and where KDE is used for visualizing data and spotting anomalies. 17 October 2026 · 7 min read Probability and statistics The law of large numbers and the central limit theorem explained Why sample averages settle toward the true mean and why their error is bell-shaped with a spread of sigma over root n, plus when neither result holds, with PyTorch simulations of each. 17 October 2026 · 7 min read Probability and statistics The multivariate Gaussian, covariance matrices, and Mahalanobis distance How to read and estimate a covariance matrix and how much a full or diagonal one costs to learn, plus how Mahalanobis distance flags outliers, with PyTorch code for each step. 17 October 2026 · 8 min read Probability and statistics Common probability distributions in ML: Bernoulli, binomial, Poisson, Beta Which distribution fits which question, the formula and mean of each one worked with real numbers, and how to check them in PyTorch. 16 October 2026 · 10 min read Probability and statistics Degrees of freedom explained Why sample variance divides by n - 1, what the degrees of freedom of Student's t and chi-squared control, and a PyTorch simulation that shows dividing by n runs low. 16 October 2026 · 5 min read Probability and statistics The Gaussian (normal) distribution explained What the mean and standard deviation do to the bell curve, how to standardize any Gaussian to z-scores, and why code works with the log-density, with every step checked in PyTorch. 16 October 2026 · 6 min read Probability and statistics Bayes' theorem explained with an example Where the formula comes from, what the prior, likelihood and posterior mean, and a spam filter example worked by hand and in PyTorch, including how the prior changes the answer. 15 October 2026 · 6 min read Probability and statistics Dependent vs independent variables in machine learning Which part of a dataset is X and which is y, the other names each one goes by (features, labels, targets, predictors), and how they look as PyTorch tensors. 15 October 2026 · 5 min read Probability and statistics Joint, marginal, and conditional probability explained, with independence How to read a joint probability table, get a marginal by adding a row, get a conditional by dividing, test two variables for independence, and factor any joint with the chain rule, with PyTorch code. 15 October 2026 · 10 min read Calculus and optimization Convex functions explained (and why neural network losses are not convex) How to check whether a function is convex with a chord, the Hessian or a short list of known shapes, what Jensen's inequality says, and what a convex optimization problem looks like, with PyTorch code for the chord and Hessian checks. 14 October 2026 · 10 min read Calculus and optimization Local minima and saddle points in deep learning How to find the points where a function's gradient is zero, tell a minimum from a maximum or a saddle with the second-derivative test, and see in PyTorch why gradient descent can stall at a saddle. 14 October 2026 · 7 min read Probability and statistics Random variables and probability distributions explained: PMF vs PDF How a die roll and a classifier's output are both described by a distribution, the two rules every distribution follows, and why a density can be larger than 1 while a probability cannot, with PyTorch code. 14 October 2026 · 5 min read Calculus and optimization Automatic differentiation explained: how autograd works How PyTorch gets the gradient of a loss for millions of parameters in one backward pass, why it walks the computation graph from the output back to the inputs, and when forward mode is the better fit, with code you can run. 13 October 2026 · 8 min read Calculus and optimization The Hessian matrix explained, and the eigenvalue test for minima How to compute the Hessian of a function by hand, why it is symmetric, and how its eigenvalues tell a minimum from a maximum or a saddle point, with PyTorch code to check each step. 13 October 2026 · 8 min read What NLP is How to learn NLP in 2026: a complete roadmap from text to LLMs What to learn for natural language processing and in what order, from tokenization and word embeddings to transformers, LLMs and post-training, so you can see what is ahead and check off what you already know. 13 October 2026 · 14 min read Calculus and optimization Taylor series for machine learning: linearization and why gradient descent works How to approximate a function near a point with its gradient and Hessian, how far you can trust that approximation, and what it says about why a small gradient descent step lowers the loss, with worked numbers and PyTorch code. 13 October 2026 · 7 min read Calculus and optimization The chain rule explained for machine learning How to differentiate a function that feeds into another function, how the rule extends to several paths and to vector-valued layers, and how to check the result in PyTorch. 12 October 2026 · 6 min read Calculus and optimization Matrix calculus for deep learning: the gradient of a linear layer, by hand A gentle walk through the few vector and matrix gradients deep learning reuses, with small examples you can check by hand, ending with the backward pass of a linear layer worked out with real numbers and checked in PyTorch. 12 October 2026 · 9 min read Calculus and optimization The Jacobian matrix explained How to build the Jacobian of a function with several inputs and outputs, why the gradient is a one-row Jacobian, and how to use it to chain layers and estimate output sensitivity, with PyTorch checks. 12 October 2026 · 5 min read Calculus and optimization Derivatives for machine learning: limits, continuity, and the derivative How the derivative is built from a limit, when a function has no derivative at a point (and why ReLU has a kink at 0), the seven rules that cover most expressions, and how to get the sigmoid's derivative in one line, with Python to check each result. 11 October 2026 · 6 min read Linear algebra Einsum explained with examples (torch.einsum) How to read an einsum string like 'bik,bkj->bij', which letters get summed and which stay, and how to write the dot product, matrix multiplication, transpose, batched matmul, elementwise product, and outer product with torch.einsum. 11 October 2026 · 6 min read Calculus and optimization Partial derivatives and gradients explained How to take the derivative of a function with several inputs one input at a time, how those derivatives stack into the gradient, and why stepping against the gradient lowers a loss fastest, with worked examples and PyTorch checks. 11 October 2026 · 6 min read Linear algebra Broadcasting in NumPy and PyTorch explained How to tell in advance whether two shapes can be combined and what shape comes out, how to make a vector stretch across rows or across columns, and how to build a table of every pairwise difference without a loop, with PyTorch code for each. 10 October 2026 · 5 min read Linear algebra SVD explained: singular value decomposition and low-rank approximation What the three factors of an SVD do to a vector, how to read a matrix's rank off its singular values, and how keeping only the largest few gives the closest possible low-rank copy of the matrix, with PyTorch code for each. 10 October 2026 · 6 min read Linear algebra What is a tensor? Shapes, batches, and tensor operations How to read a tensor's shape, why models process a whole batch with one matrix multiplication, and when to use reshape, squeeze, sum and permute, with PyTorch code for each and the reshape mistake that silently scrambles attention heads. 10 October 2026 · 5 min read Linear algebra The condition number explained: why some problems are numerically unstable How to compute the condition number of a matrix from its eigenvalues, what the Rayleigh quotient says about the largest and smallest eigenvalue, and two PyTorch experiments that show a large condition number amplifying small errors and slowing gradient descent. 9 October 2026 · 5 min read Linear algebra Cosine similarity explained: the angle between two vectors How to compute cosine similarity by hand and in PyTorch, what scores of 1, 0 and -1 mean, and when to compare embeddings with it instead of the raw dot product. 9 October 2026 · 5 min read Linear algebra The dot product explained, and what it measures in machine learning How to compute a dot product by hand, what its sign says about two vectors, how it relates to the angle between them, and the three PyTorch calls that compute it. 9 October 2026 · 5 min read Linear algebra Eigenvalues and eigenvectors explained How to find the eigenvalues and eigenvectors of a matrix, how they relate to its determinant and trace, how a change of basis rewrites a vector or a matrix, and how eigendecomposition turns a matrix into per-axis scaling, with PyTorch code throughout. 9 October 2026 · 6 min read Linear algebra Invariance vs equivariance, and why convolution is translation equivariant The two equations that separate an invariant function from an equivariant one, worked examples with rotations and shifts, which network layers have which property, and PyTorch checks you can run on a convolution, a pooling step, and a shuffled hidden layer. 9 October 2026 · 4 min read Linear algebra Least squares explained: linear regression as a projection, and the normal equation How least squares becomes a projection onto a column space, how the normal equations come out of one orthogonality condition, what the Gram matrix is and when it fails, and how to fit a line in PyTorch. 9 October 2026 · 5 min read Linear algebra Orthogonality explained: orthogonal matrices, orthonormal bases, and projections How to tell when two vectors or a whole matrix are orthogonal, why the inverse of an orthogonal matrix is its transpose, how to project one vector onto another, and how Gram-Schmidt builds an orthonormal basis, with PyTorch code for each. 9 October 2026 · 7 min read Linear algebra Positive definite matrices explained, with the Cholesky and QR factorizations How to test whether a matrix is positive definite, semi-definite, negative definite or indefinite, why every covariance matrix passes the semi-definite test, and when to use the QR and Cholesky factorizations, with PyTorch code for each. 9 October 2026 · 6 min read Linear algebra Vector norms and distances: L1, L2, Euclidean vs Manhattan How to compute the L1, L2 and L-infinity norms by hand, when Euclidean and Manhattan distance give different answers, why L1 and L2 penalties shrink weights differently, and the PyTorch calls for each. 9 October 2026 · 6 min read Linear algebra Determinant, trace and inverse: when a matrix is singular How to compute a determinant for 2x2 and 3x3 matrices, what a zero determinant means for inverting a matrix and solving equations, why torch.linalg.solve beats torch.linalg.inv in code, and where the trace shows up. 8 October 2026 · 7 min read Linear algebra Solving systems of linear equations with Gaussian elimination How to solve a 2x2 and a 3x3 system of equations by hand with row operations, how to tell from the reduced rows whether there is one solution, none, or infinitely many, and the same steps in PyTorch. 8 October 2026 · 7 min read Linear algebra Hadamard product vs matrix multiplication: * vs @ in PyTorch The Hadamard product multiplies two matrices position by position, which is what * does in PyTorch, while matrix multiplication combines rows with columns, which is what @ does. This post compares the two on the same numbers, explains the shape rules for each, and shows where the Hadamard product is used in models. 8 October 2026 · 5 min read Linear algebra Matrix multiplication explained: the linear map view, dot products and outer products Matrix multiplication combines two grids of numbers by pairing the rows of one with the columns of the other. This post explains how to compute it by hand, why the shapes have to line up, how a matrix transforms vectors, and two other ways to read the same product, with PyTorch code for each. 8 October 2026 · 7 min read Linear algebra Multicollinearity explained: why correlated features break linear regression Multicollinearity is when two or more features in a dataset carry almost the same information. This post explains why that makes a linear model's weights unstable even when its predictions look fine, shows it happening in PyTorch, and covers what to do about it. 8 October 2026 · 5 min read Linear algebra Rank, column space and null space explained How to find the rank of a matrix by hand, what the column space and null space say about whether Ax = b has a solution, why a large weight matrix can hold far fewer independent directions than its size, and how to check all of it in PyTorch. 8 October 2026 · 6 min read Math toolkit and notation Big-O notation for machine learning Big-O notation describes how the work an algorithm does grows as its input gets bigger. This post explains what it keeps and what it drops, the growth rates you will meet (O(1), O(log n), O(n), O(n log n), O(n²)), and why an all-pairs method that is fine on a thousand items fails on a million. 7 October 2026 · 3 min read Math toolkit and notation Exponents and logarithms for machine learning Logarithms show up in most machine learning loss functions because a log turns a product into a sum. This post covers the exponent and log rules, what an inverse function is, and why training works with log-likelihood instead of multiplying probabilities, with PyTorch code. 7 October 2026 · 6 min read Math toolkit and notation How to read the math notation in ML papers Machine learning papers write their formulas with a small set of symbols: big sigma and pi, argmin and argmax, hats and bars over letters, a left arrow, and shape declarations like W ∈ ℝ^(m×n). This post explains what each one means and shows the PyTorch line that does the same thing. 7 October 2026 · 7 min read Math toolkit and notation Sine and cosine for machine learning Sine and cosine are the coordinates of a point on a circle of radius 1, and they repeat every full turn. This post explains the unit circle, the identity sin² + cos² = 1, and how sine and cosine features let a model see that hour 23 and hour 0 are one hour apart, with PyTorch code. 7 October 2026 · 4 min read Linear algebra Span, basis and linear independence explained The span of a set of vectors is every point you can reach by scaling and adding them, a basis is the smallest set that reaches a whole space, and linear independence says no vector in the set is redundant. This post explains all three with 2D examples, what they mean for features in a dataset, and how to check them in PyTorch. 7 October 2026 · 4 min read Linear algebra Vectors and matrices for machine learning Machine learning stores data as scalars, vectors, matrices and tensors. This post explains each one, how shapes and indexing work in PyTorch, adding and scaling vectors, the transpose, and the identity, diagonal and symmetric matrices. 7 October 2026 · 5 min read Evaluation A step-by-step workflow for improving a machine learning model When a model trains but its results are not good enough, a five-step loop tells you what to change next: pick a metric tied to the goal, build a small baseline, read the gap between training and test error, make one targeted change, and measure again. 6 October 2026 · 7 min read Evaluation ML model monitoring after deployment: data drift and when to retrain A model that tested well can get worse after launch because the data it sees keeps changing while the model stays the same. This post explains how that happens, how to notice it, and how to retrain and replace a live model safely in PyTorch. 6 October 2026 · 6 min read Evaluation How to measure model size and speed: parameters, FLOPs, latency, throughput, memory Accuracy says how good a model's predictions are, not whether it is small and fast enough to run. This post shows how to measure parameter count, FLOPs, latency, throughput, and memory in PyTorch, and which one to look at for which job. 6 October 2026 · 8 min read Evaluation How to choose an evaluation metric for your model Accuracy, precision, recall, ROC AUC and R² each miss a different kind of failure. This post shows what each one misses, how to pick one primary metric from the cost of your model's mistakes, and which two or three metrics to track next to it. 5 October 2026 · 7 min read Evaluation Loss is NaN: the common causes and fixes A loss that turns into NaN usually comes from a log of zero, an exponential that overflows, or a division by zero. This post shows how to catch the NaN on the step it appears, find the layer that produced it, and fix each common cause in PyTorch. 5 October 2026 · 8 min read Evaluation Regression metrics: MAE, RMSE, R², and MAPE MAE, RMSE, R² and MAPE are the usual numbers for saying how good a model that predicts prices, sales or other quantities is. This post explains what each one measures, how to compute them in PyTorch, and which one to report for your problem. 5 October 2026 · 8 min read Evaluation Cross-validation explained: k-fold, stratified, and when not to use it A score from one train/validation split can swing a lot depending on which examples landed where. This post explains k-fold and stratified k-fold cross-validation, builds both in plain PyTorch, and says when a single split or a time-ordered split is the better choice. 4 October 2026 · 5 min read Evaluation Data leakage in machine learning: examples and how to prevent it A model that scores well during development can still fail on new data when its evaluation was contaminated. This post shows common examples of data leakage, how to spot it on a loss curve, and how to prevent it in PyTorch. 4 October 2026 · 7 min read Generalization and regularization Hyperparameter tuning: grid search vs random search Picking the learning rate and layer sizes by trial and error wastes training runs. This post compares grid search, random search, and successive halving, with runnable PyTorch code and advice on which to use. 4 October 2026 · 7 min read Evaluation Precision, recall, and F1 score explained, with the confusion matrix Precision, recall, and F1 score measure how well a classifier finds the cases you care about, which accuracy can hide. This post explains the confusion matrix they come from, what each metric tells you, how the decision threshold trades one for the other, and how to compute them in PyTorch. 4 October 2026 · 11 min read Evaluation ROC curve and AUC explained, and when to use PR AUC instead The ROC curve and its AUC score tell you how well a classifier ranks positive examples above negative ones, without picking a threshold. This post explains how the curve is built, what an AUC of 0.5 or 0.9 means, why it can look good on rare-class data, and when to report PR AUC instead. 4 October 2026 · 7 min read Generalization and regularization Label smoothing explained Label smoothing is a small change to classification training that stops a model from becoming 100% sure of its answers. This post explains what it changes in the targets, why that helps, how to set it in PyTorch, and when to leave it off. 3 October 2026 · 5 min read Generalization and regularization Weight decay vs L2 regularization: why they differ under Adam Weight decay and L2 regularization both pull a model's weights toward zero, and with plain gradient descent they are the same thing. This post explains what each one does, why they stop being the same under Adam, and why AdamW is the usual fix. 3 October 2026 · 6 min read Generalization and regularization What is dropout and why does it work? Dropout is a regularization technique that randomly switches off part of a neural network during training so the model does better on new data. This post explains what dropout does on each step, why it reduces overfitting, how it behaves at test time, and how to use it in PyTorch. 3 October 2026 · 5 min read Generalization and regularization The curse of dimensionality explained The curse of dimensionality is what happens to a dataset when you add input features without adding examples. This post shows why the same data covers less of the input space with each new feature, why distances between examples stop being useful, and what to do about it. 2 October 2026 · 7 min read Generalization and regularization Double descent explained Double descent is what happens when a model keeps growing after it can fit its training data exactly. Its error on new data rises to a peak and then falls a second time. This post explains where the peak sits, why very large models can do better than mid-sized ones, and how to reproduce the curve in a few lines of PyTorch. 2 October 2026 · 7 min read Generalization and regularization How much data do you need to train a model? The amount of training data a model needs depends on how large the model is, how accurate it has to be, and how noisy the data is. This post explains that relationship, the formal version from learning theory, and a test that tells you whether more data would help your model. 2 October 2026 · 8 min read Generalization and regularization How to read a loss curve: training vs validation loss A loss curve is a plot of a model's loss over the course of training. This post explains how to plot training and validation loss together, what a healthy curve looks like, and what it means when validation loss goes up while training loss keeps going down. 1 October 2026 · 10 min read Optimization How to make PyTorch training reproducible: seeds, determinism, and what still varies Reproducible training means running the same code on the same data and getting the same result. This post explains where the randomness in a PyTorch run comes from, how to seed each source, how to seed a DataLoader, and which differences remain after seeding. 1 October 2026 · 8 min read Optimization How to save and resume training in PyTorch (checkpoints) A checkpoint is a file that stores a training run so it can be continued later. This post explains what to put in one, how to save and load it in PyTorch, why saving only the model weights is not enough to resume, and why most runs keep a latest and a best checkpoint. 1 October 2026 · 9 min read Optimization Batch vs stochastic vs mini-batch gradient descent, and how to choose a batch size Batch, stochastic, and mini-batch gradient descent differ only in how many training examples each update looks at. This post explains what that changes, why mini-batch is the usual choice, and how to pick a batch size and adjust the learning rate with it. 30 September 2026 · 7 min read Optimization Learning rate schedules explained: warmup, cosine annealing, step decay, and ReduceLROnPlateau A learning rate schedule changes the step size of training as the run goes on, usually large at the start and small at the end. This post explains why a fixed learning rate falls short, how linear warmup, cosine annealing, step decay, and ReduceLROnPlateau work, and how to set each one up in PyTorch. 30 September 2026 · 12 min read Optimization Train, validation, and test sets: what each is for and how to split A dataset is usually split into three parts: a training set the model learns from, a validation set for making choices, and a test set for the final score. This post explains what each split is for, why the test set is kept out of model choices, and how to split data in PyTorch. 30 September 2026 · 7 min read Neural networks Dying ReLU: what dead neurons are and how to fix them A dead ReLU neuron outputs zero for every input and stops learning, and a network can lose a large share of its neurons this way during training. This post explains why ReLU neurons die, how to measure the dead fraction in PyTorch, and how to prevent it with initialization, the learning rate, and Leaky ReLU. 29 September 2026 · 8 min read Neural networks Vanishing and exploding gradients, and how gradient clipping helps Vanishing and exploding gradients happen when the training signal shrinks to almost nothing or grows out of control as it passes back through a deep network. This post shows why both happen, how to measure them in PyTorch, and how ReLU, good initialization, and gradient clipping help. 29 September 2026 · 7 min read Neural networks Weight initialization explained: Xavier vs He (Kaiming) vs orthogonal Weight initialization is how a neural network's weights are set before training starts, and a bad choice can stop a deep network from learning at all. This post shows what goes wrong with zero, too large, and too small weights, and when to use Xavier, He (Kaiming), or orthogonal initialization in PyTorch. 29 September 2026 · 8 min read Neural networks Parameters vs hyperparameters in machine learning Parameters are the numbers a model learns from data, and hyperparameters are the settings you choose before training starts. This post explains the difference with a small PyTorch network, shows where each one lives in code, and how to pick hyperparameters with a validation set. 28 September 2026 · 6 min read Neural networks The universal approximation theorem explained (and what it does not promise) The universal approximation theorem says a neural network with one wide enough hidden layer can match any continuous curve as closely as you like. This post explains what that means, why it works, and the four things the theorem does not promise about training and real data. 28 September 2026 · 6 min read Neural networks What is a neural network? The multilayer perceptron explained A neural network passes its input through layers of simple units, each layer reshaping the numbers for the next one. This post explains the multilayer perceptron layer by layer, why stacking layers helps, and how to count its parameters by hand and in PyTorch. 28 September 2026 · 8 min read How a model learns Logistic regression explained: sigmoid, binary cross-entropy, and why it is a one-layer neural network Logistic regression predicts the probability that an input belongs to one of two classes. This post explains the sigmoid function, the binary cross-entropy loss it trains with, why the two are paired, and how to train it in PyTorch. 27 September 2026 · 9 min read How a model learns Multi-class vs multi-label classification: softmax vs sigmoid outputs Multi-class classification picks exactly one label per input, and multi-label classification can pick several or none. This post explains the difference, why the first uses softmax and the second uses one sigmoid per label, and how to write both in PyTorch. 27 September 2026 · 5 min read How a model learns Softmax explained: probabilities, model confidence, and the stable softmax trick Softmax turns a classifier's raw scores into probabilities that add up to 1. This post explains the formula with a worked example, how to read its output as the model's confidence and when not to trust it, and why softmax code subtracts the largest score first. 27 September 2026 · 7 min read How a model learns Entropy, cross-entropy, and KL divergence explained Cross-entropy is the usual loss for training a classifier. This post explains where it comes from, starting with entropy and ending with KL divergence, works each one out on the same small example, and shows how to compute them in PyTorch. 26 September 2026 · 8 min read How a model learns Maximum likelihood estimation explained: where MSE and cross-entropy come from Maximum likelihood estimation picks the model parameters that make the observed data most probable. This post explains it with a coin-flip example, then shows how the same idea produces mean squared error for regression and cross-entropy for classification. 26 September 2026 · 8 min read How a model learns The PyTorch training loop explained, with Dataset and DataLoader The training loop is the five lines of PyTorch code that make a model learn from data. This post explains what each line does, why their order can't be swapped, and how a Dataset and DataLoader feed the loop one batch at a time. 26 September 2026 · 8 min read How a model learns Linear regression from scratch in PyTorch Linear regression predicts a number, like a house price, as a weighted sum of the inputs plus a bias. This post builds it in PyTorch two ways: with gradients computed by hand, and with nn.Linear and an optimizer. 25 September 2026 · 6 min read How to learn ML Learning ML with Python: the libraries you need (NumPy, pandas, scikit-learn, PyTorch) and what each is for Four Python libraries come up when you start machine learning. This post explains what NumPy, pandas, scikit-learn and PyTorch each do, which one to learn first, and when you need the other three. 25 September 2026 · 6 min read How a model learns What is a perceptron? From the original perceptron to the modern neuron A perceptron is the simplest artificial neuron: it weighs its inputs, adds them up, and outputs 0 or 1. This post explains how it works, how it learns, why modern neurons output a real number instead, and why one neuron can never solve XOR. 25 September 2026 · 7 min read Interviews and careers Machine learning interview questions for beginners, with answers About 30 machine learning questions that come up in junior ML, data science, and ML engineering interviews, each with a short answer and the follow-up an interviewer usually asks next, grouped from data and evaluation through training, metrics, neural networks, and architectures. 24 September 2026 · 19 min read How a model learns What is machine learning? The 5 types of learning explained Machine learning is a way to build software that learns its rules from examples instead of having them written by hand. This post explains how that works and why it can work on data the model has never seen, then walks through the five types of learning (supervised, unsupervised, semi-supervised, self-supervised, and reinforcement learning) with an example of each and how to pick between them. 24 September 2026 · 8 min read Interviews and careers How to choose a machine learning project A good machine learning project usually starts from a problem you or someone close to you actually has. This post covers where to find one, the questions to ask before you start, and what a project needs to be worth showing: a baseline, more than one dataset, more than one model, and more than one way to evaluate it. 23 September 2026 · 7 min read How a model learns What is a loss function? MSE, cross-entropy, and when to use which A loss function turns a model's predictions into one number that says how wrong they are, and training is the process of pushing that number down. This post explains why models need one, which loss to use for regression, binary, multi-class, and multi-label problems (and for a few other tasks), and how to use the loss on a proper validation split to evaluate a model. 23 September 2026 · 13 min read Neural networks Activation functions roadmap: step, sigmoid, tanh, ReLU, Leaky ReLU, GELU An activation function is the small non-linear function each neuron applies to its output. This post goes through the common ones in the order they were invented, what each fixed, what each broke, and how to write every one in PyTorch. 22 September 2026 · 6 min read Generalization and regularization Bias-variance tradeoff explained with pictures A model can be wrong on new data in two different ways, and the fixes for each pull in opposite directions. This post shows both with plots, how to tell them apart from your training and validation loss, and what to change for each. 22 September 2026 · 7 min read Interviews and careers What ML projects actually get you hired in 2026? The machine learning projects that help in hiring are the ones you cared enough about to finish, measure, and defend. This post covers how to pick one, what a reviewer checks, five projects worth building, how to present it, and how to answer when an interviewer asks why you didn't do it another way. 21 September 2026 · 7 min read Generalization and regularization Regularization explained: L1, L2, dropout, and early stopping Regularization is a set of techniques that stop a model from memorizing its training data so it does better on new data. This post explains four common ones (L2 weight decay, L1, dropout, and early stopping), how each one works, and how to set them in PyTorch. 21 September 2026 · 6 min read How a model learns What is backpropagation? Explained with a tiny network Backpropagation is how a neural network works out which way to change each of its weights to reduce its error. This post computes it by hand on a network with two weights, shows why it runs backward, and checks the numbers against PyTorch. 21 September 2026 · 5 min read How to learn ML How much math do you actually need for machine learning? You need enough math to read a loss function, an update rule, and a shape error, and that is a list you can finish in a few weeks. This post names every piece, says which ones you need before your first model and which can wait, and estimates the hours. 17 September 2026 · 5 min read Optimization Machine learning optimizers explained: SGD, momentum, RMSProp, Adam, AdamW An optimizer is the rule that turns gradients into parameter updates. This post explains the five optimizers most models train with, in the order each one was invented to fix the last, and which one to pick. 17 September 2026 · 7 min read Generalization and regularization Overfitting vs underfitting: how to tell which one you have, and how to fix it Overfitting is when a model memorizes its training data and fails on new data. Underfitting is when it cannot even fit the training data. This post shows how to read which one you have from the loss curves and what to change for each. 17 September 2026 · 6 min read How to learn ML How to learn machine learning effectively (and actually remember it) Most people finish an ML course and cannot use it a month later. This post covers what to learn in which order, how to practice so the concepts stay, and a weekly schedule you can keep with 30 to 45 minutes a day. 16 September 2026 · 7 min read Evaluation My loss is not decreasing: a checklist A training loss that stays flat usually has one of a short list of causes. This checklist goes through them in a practical order: the data, the training loop, the learning rate, the loss function, initialization, and a one-batch test that separates a bug from a hard problem. 16 September 2026 · 7 min read Optimization What is a learning rate, and how do you pick one? The learning rate is the number that most often decides whether a model trains well. This post explains what it does inside the update rule, how to read the three shapes of a loss curve, why good values differ between SGD and Adam, and a five-line test that finds a usable value. 16 September 2026 · 6 min read How a model learns What is gradient descent? Explained step by step Gradient descent is the procedure that trains almost every machine learning model. This post explains what it does, why the update has a minus sign, how the learning rate changes everything, and what the three lines of PyTorch that implement it are doing. 16 September 2026 · 6 min read Optimization What is the Adam optimizer? Explained step by step Adam is the optimizer most deep learning models train with. This post explains what it does, why it works better than plain gradient descent, and when to use AdamW instead. 15 September 2026 · 5 min read How to learn ML How to learn machine learning in 2026: a complete roadmap for self-taught learners The order to learn machine learning on your own: Python, the math that ML uses, PyTorch, then training and evaluating deep networks, with every concept named so you know exactly what is ahead of you. 15 September 2026 · 14 min read