The Computer That Learns Exactly Like You Do (and Fails the Same Way First)

Picture a neural network on its first day trying to recognise a cat. It looks at a photograph and makes a guess. Wrong. It looks at another one. Wrong again, probably a fish. Nobody has sat down and written it a list of cat rules – four legs, furry, expression of mild contempt for everyone in the room. It has no rules. It just guesses, fails, gets told it failed, and adjusts. Billions of tiny adjustments, until the pattern clicks.

Backpropagation, Briefly

This is called backpropagation, which is a marvellous word to say aloud and then immediately forget. What it describes is simpler: every time the network gets something wrong, the error travels backwards through all the tiny connections that led to that guess, nudging each one very slightly in the right direction. It does not cross out its answer and copy the correct one. It fails in a particular direction, and that direction tells it something.

Here is where it gets odd. This is almost exactly what happens in your own head.

What Happens When You Try to Remember

When you try to remember something – a name, a fact, a French irregular verb – and you can't quite get it, your brain is not just passively waiting. It is doing something active. Searching, reaching, misfiring, correcting. The struggle itself changes the physical structure of the connection. The neuroscience term is long-term potentiation, which is also a lovely thing to forget. What it means is that retrieving information, especially when it is effortful, rewires the brain more durably than being told the answer does. Your failures carve the path. Getting something wrong, and then getting it right, is more powerful than getting it right the first time with help.

Researchers working on AI and researchers working on human memory have started using almost identical language to describe these two systems. Both talk about "training". Both talk about "error signals". Both describe a process that makes no progress without encountering resistance. For the link between the two ideas, read The Test Isn't the End of Learning. It's Where Learning Actually Happens at https://budgetingtips.co.uk/education/the-test-isn-t-the-end-of-learning-it-s-where-learning-actually-happens/ which takes this idea somewhere even more interesting from the human side.

The Upshot for AI

The upshot for AI is also quietly strange. Modern language models – the kind behind chatbots and spam filters and film recommendations – are not smarter versions of encyclopaedias. They were not given facts and told to store them. They were fed enormous quantities of text and trained by repeatedly predicting what word comes next. Getting it wrong. Adjusting. Getting it wrong slightly differently. Adjusting again. Billions of cycles of exactly this. The cat recogniser, the spam filter, the next-word guesser: all of them learned by failing, on purpose, at scale.

Which means the thing that makes a machine intelligent and the thing that makes a human better at remembering are, underneath the vocabulary, embarrassingly similar. Both need the test. Not as proof of what they know. As the actual mechanism by which they get there.