AI Learns Like a Toddler: The Training Loop and 70 Billion Dials
Strip away the math and a language model learns the way a toddler learns to talk: guess, check, adjust. It guesses the next word, gets told how wrong it was, and adjusts slightly so the same mistake hurts a little less next time. That's the whole trick — repeated so many times, at such scale, that "slightly less wrong" compounds into fluency.
The training loop
Training is one loop, run over trillions of words of text:
- Predict. Given "the cat sat on the ___", the model outputs a probability for every possible next token.
- Compare. The real next word is in the data. The gap between the model's guess and the truth is computed as an error score (the "loss").
- Nudge. Every internal weight that contributed to the error gets adjusted a tiny amount in the direction that would have made the guess better.
- Repeat. Billions upon billions of times, across the whole dataset, on thousands of chips, for weeks or months.
No single pass teaches the model anything you'd notice. The intelligence is entirely in the accumulation.
What "parameters" actually are
When you hear "a 70-billion-parameter model", those parameters are the dials the loop was nudging. Each one is just a number — a weight in the network — and the training process tuned all 70 billion of them, together, until the model's next-word guesses got good. The finished set of dial positions is the model's accumulated knowledge: grammar, facts, code idioms, reasoning patterns, all encoded as nothing more than 70 billion carefully-set numbers. There's no database of facts inside — just the dials.
Why you should care: trained means frozen
Here's the part that changes how you use these tools day to day. Once training ends, the dials stop moving. When you chat with a deployed model, it is not learning from you. Your corrections, your preferences, the project context you explained yesterday — none of it updates a single weight. At inference time the model is a read-only artifact; each conversation starts from the same frozen state.
When an AI product does "remember" you, that's an explicit feature built around the model — saved notes injected back into the context window, a memory store you opted into — not the model itself learning. Understanding that distinction explains most "why did it forget?" frustration: the model never knew; the product either re-supplied the context or it didn't.
Power tricks
- Restate key context each session. Project constraints, style rules, definitions — repeat them or keep them in a reusable prompt. The model starts cold every time.
- Use the memory features you're given. Custom instructions, project files, memory settings — these are the sanctioned way to persist context. Opt in deliberately and audit what's stored.
- Don't assume it remembers. If an answer depends on something you said last week, paste it again. "It should know this by now" is never true at the weight level.
Resources
Building an AI feature? Yeda AI designs, audits, and ships production LLM systems.