Mathematics: descent, description, chance, and what counts as a mistake
Two ideas carry most of the mathematics you need. The first is gradient descent: step downhill on an error surface until you reach a low point. Change the learning rate and watch it crawl, converge, or bounce out entirely — that one control explains a large share of training failures. The second is describing a distribution honestly: mean, median, spread and shape, and why the mean stops being a useful summary the moment the data is skewed. Three more panels follow. A test that is 99% accurate, applied to a condition almost nobody has, and the number that matters to the person holding the positive result — which is nothing like 99%, because the healthy majority produces more false alarms than there are real cases. Then sampling: skew a population as far as you like, take averages of samples from it, and watch those averages pile into a bell anyway and tighten as the sample grows — four times the sample, half the spread, which is why a poll of a thousand can say anything about millions. And finally what counts as a mistake: two lines fitted to the same points, one charging the square of each error and one charging the error, with a single outlier you can drag. Only one line chases it, and the one that chases is the default in almost every library you will use. Two more sit underneath all of it. Correlation compresses a whole scatter into one number — and the panel shows the three ways that number lies: a perfect curve scoring zero, and pure noise with a single stray reading scoring 0.83, the same r as a genuinely strong relationship. And a matrix, which is four numbers until you apply it to every point in a grid and it becomes a movement — a stretch, a shear, a rotation. Push it until the determinant reaches zero and the whole plane collapses onto a line, which is the one thing no later matrix can undo. You do not need to derive anything here. You need to see what the numbers do when you move them, so the formulas you meet later describe something you have already watched happen.
Calculus
Gradient descent
Training is just rolling downhill. At each step the model moves against the slope of the loss. The learning rate sets the step size — too big and it overshoots, too small and it crawls.
- Set a very small learning rate and run. It creeps downhill and never arrives. Correct direction, useless in practice — this is a model that trains for a week and learns nothing.
- Raise it until the steps just stop overshooting. It reaches the bottom in a handful of steps. That narrow band is what people are hunting for when they tune this.
- Push it higher still. The steps grow instead of shrinking and it leaves entirely. Nothing is broken — the rate is simply too large for the slope.
A moderate learning rate slides smoothly to the minimum.
Probability & statistics
Describe your data
Click anywhere on the line to drop a data point. Mean, median, mode, and standard deviation update instantly — see how a few outliers pull the mean away from the median.
- Add a handful of points close together. Mean and median sit almost on top of each other. On symmetric data either one summarises it.
- Now add one point far to the right. The mean chases it; the median barely moves. One salary, one outage, one invoice can move an average that a whole team then plans against.
- Add several more out there. Now the median moves too. It resists one oddity, not a genuine second group — that is a different problem.
When mean and median drift apart, your data is skewed by outliers.
Probability
A 99% accurate test, and the answer that shocks people
Set how common the condition is, and how good the test is at spotting it and at clearing the healthy. The square is ten thousand people. Then read the only number that matters to someone holding a positive result: of everyone the test flagged, what share actually has it?
Out of every 10,000 people.
- Leave the test at 99% and 99%, and set the condition to 10 in 10,000. A positive result means about 9%. Ten real cases, a hundred false alarms — the test is excellent and the answer is still mostly wrong.
- Raise how common the condition is to 500 in 10,000. The same test now gives about 84%. Nothing about the test changed — only who was tested.
- Go back to 10, and raise only the specificity. This is the dial that matters when the condition is rare — clearing the healthy, not catching the ill.
When the condition is rare, the healthy majority produces more false positives than there are true cases — however good the test is. The test's accuracy is not the answer to the question the patient is asking.
Statistics
Why averages behave when nothing else does
The top row is a population — skew it as far as you like. The bottom is what you get by taking samples of a chosen size and plotting the average of each one. Draw samples and watch the second shape build.
From heavily skewed to symmetric.
How many people in each sample.
- Skew the population hard, set the sample to 1, and draw 1,000. The bottom shape copies the top one. A sample of one is just a point from the population.
- Leave the skew, set the sample to 10, and draw again. A bell appears underneath a lopsided population. This is the result the whole of inferential statistics rests on.
- Now compare a sample of 10 with a sample of 40. The spread roughly halves — four times the sample, half the error. That is why polls are the size they are.
The averages pile into a bell however lopsided the population is, and they tighten as the sample grows — four times the sample halves the spread. That is why a poll of a thousand can say anything at all about millions.
The mathematics of fitting
What counts as a mistake
Both lines fit the same points; they disagree about how much a big miss should cost. Squared error charges the square of the distance, absolute error charges the distance. Drag the outlier and watch only one of them chase it.
How far one point sits from the rest.
- Leave the outlier where it starts. The two lines sit on top of each other. With well-behaved data the choice of loss makes almost no difference.
- Drag it all the way up. The squared-error line lifts towards it; the absolute-error line stays with the crowd. One point, one line moved.
- Ask which line you would want. It depends on whether that point is an error or the most important thing in the data. Squared error is the default, and it assumes the second.
Squared error is the default in almost every library, which means one bad row can move your model. That is a choice somebody made for you — and the reason a median survives an outlier where a mean does not.
Statistics
One number for a relationship, and what it hides
Correlation compresses a whole scatter into a single number between -1 and 1. Move the relationship and the noise, and watch r respond — then try the shapes where it stops telling the truth.
- Keep the shape straight and raise the strength. r climbs towards 1 as the cloud tightens onto a line. This is the case everyone pictures when they hear the word.
- Now switch the shape to curved. A tight, obvious, perfectly predictable relationship — and r near zero. It only ever measured straight lines.
- Switch to noise plus one outlier. There is no relationship at all, and r is respectable. One row can manufacture a finding — which is why you look at the scatter before you quote the number.
r only measures STRAIGHT-LINE agreement. A perfect curve can score near zero, and one far-off point can manufacture a relationship out of noise. The number is never a substitute for the picture.
Linear algebra
A matrix is a thing you do to space
Four numbers in a square do not look like much. Apply them to every point in a grid and they become a movement — a stretch, a rotation, a shear. Move the four and watch the shape follow.
- Leave it at no change, then raise a alone. The square stretches sideways and the determinant tracks it. Each number moves one corner of the grid.
- Set b to about 1 and leave the rest. The square leans into a parallelogram — a shear. Nothing rotated and nothing changed size; the space slid.
- Now make the two rows proportional — try a=1, b=2, c=1, d=2. The determinant hits zero and the grid collapses onto a single line. Two dimensions became one, and no matrix can undo it.
The determinant is the area the unit square becomes. When it hits zero the whole plane has been flattened onto a line — and nothing you do afterwards can get the lost dimension back. That is what a singular matrix is.