Ask a chatbot to explain quantum tunneling, draft a resignation letter, or talk you down after a rotten day. It’ll do all three in fluent, confident prose. It feels like talking to something that gets you. That feeling is the whole problem. Underneath the conversation, the system isn’t reaching for meaning — it’s calculating which word is likely to come next. So how do large language models actually work, and does that machinery add up to anything we’d recognize as understanding?
The honest answer is that the mechanism is simpler than the marketing suggests. The philosophy is messier than either the boosters or the skeptics admit. Let’s take both apart.
How do large language models actually work, in one sentence?
Here it is. A large language model is a neural network — a transformer — trained to guess the next token in a stretch of text.
That’s the trick, stripped bare. When you send a prompt, the model breaks it into tokens and converts those tokens into numbers. Then it calculates a probability across its entire vocabulary for what comes next. It picks one. Then it does the whole thing again, with your prompt plus that new token. And again, until the reply is done. As IBM puts it, the model “does not ‘know’ the final answer in advance.” Instead it uses every statistical relationship it learned in training to predict one token at a time. There’s no lookup table. Each word is a fresh guess informed by everything the model absorbed in training.
The surprising part is scale. GPT-3 kicked off the current wave in 2020. It held 175 billion parameters and trained on roughly 499 billion tokens of text. Meta’s open Llama 3.1, released in 2024, runs 405 billion parameters trained on more than 15 trillion tokens. That run used over 16,000 NVIDIA H100 GPUs. Parameters are the adjustable dials the model tunes during training. For scale, the human brain carries about 86 billion neurons but an estimated 100 trillion synaptic connections. And synapses, not neurons, are the closer analogy to parameters. Frontier models are approaching that order of structural complexity. They are not replicating how a brain computes.
One telling shift: the top labs have mostly stopped bragging about size. Current GPT, Gemini and Claude releases don’t publish parameter counts. Reviewers now compare them on capability and price rather than raw numbers. “Bigger number wins” stopped being the pitch.
How they learn: tokens, prediction, and autocomplete at planetary scale
Before a model can predict anything, text has to become math. A tokenizer chops your sentence into tokens — usually sub-word chunks, not whole words. Each token then maps to a numeric embedding vector, the actual input the network computes on. “Understanding” the word cat is really the model working with a long list of numbers that stands in for cat.
Training itself is almost embarrassingly plain. The model reads mountains of raw text and practices one task: predict the next token given everything before it. Nobody hand-labels the data. Nobody attaches a “this is true” or “this is meaning” tag. IBM calls this self-supervised pretraining, and the objective never mentions truth once. The model is graded only on whether its guess matched the word that actually came next.
Think of it as autocomplete cranked up by orders of magnitude in data and model size. That framing sounds deflating until you sit with what fell out of it. Researchers at Georgetown’s CSET note the genuine surprise of the last decade. A task this dumb produced translation, coding, and reasoning-like behavior that nobody explicitly trained in. Want the wider family tree these models grew from? Our ANN vs CNN vs RNN vs GNN cheat sheet covers the architectures this one builds on.
None of this is cheap. Sam Altman has said GPT-4’s training cost more than $100 million. Independent analysis finds frontier training costs growing roughly 2.4x a year. They’re on pace to top $1 billion per run by 2027. The energy footprint scales with the price tag. Stanford’s AI Index put a single 2025 training run at over 72,000 tons of CO2-equivalent. The report’s authors stress these are inferred figures, not disclosed ones.
Inside the transformer: the trick that made 2017 a turning point
For years, language models read text the way you’d expect — left to right, one step feeding the next. Those recurrent networks struggled to connect words far apart in a sentence, and they were painfully slow to train.
Then came the 2017 paper “Attention Is All You Need.” It threw out recurrence entirely and replaced it with self-attention. In plain terms: every token looks at every other token at once. Each one decides how much the others matter for the current word. Read “the animal didn’t cross the street because it was too tired.” The model has to link it to animal, not street. Attention is how it weighs that. NVIDIA calls it detecting “subtle ways even distant data elements in a series influence and depend on each other.” It now powers nearly every serious model in the field.
Two things made this a turning point. Attention captures long-range links better than the old designs. And the model processes every token in parallel, not one after another. That’s what let training finally scale to today’s enormous models.
Embeddings carry more than raw identity. They place words in a space where similar concepts cluster together and relationships become geometric. The famous illustration: king − man + woman lands near queen. Treat that example gently, though. Careful replication shows the tidy arithmetic mostly evaporates once you stop excluding the input words from the answer. It’s a teaching device, not proof of clean human-style concepts.
Here’s where the “just autocomplete” story starts to strain. In 2025, Anthropic built interpretability tools it calls an “AI microscope” and watched a model from the inside. It found Claude planning several words ahead rather than working purely token by token. Writing poetry, it picks a rhyming target word first, then composes the line to reach it. Asked “what’s the capital of the state containing Dallas,” it activated an internal “Dallas is in Texas” step. Then it chained to “the capital of Texas is Austin.” That’s multi-hop reasoning, not a memorized string.
From raw model to chatbot: why GPT, Gemini and Claude feel different
A freshly pretrained model is not the polite assistant you talk to. Left raw, it’s a next-token machine. It’ll happily continue your prompt with more of the internet, tangents and toxicity included. Turning it into ChatGPT or Claude takes a second stage.
The workhorse is Reinforcement Learning from Human Feedback. Human raters rank several model answers to the same prompt from best to worst. Those rankings train a separate reward model, which then fine-tunes the base model toward outputs people actually preferred. The effect is startling. OpenAI’s InstructGPT proved the point. A 1.3-billion-parameter model tuned with human feedback beat the raw 175-billion-parameter GPT-3 in human preference. GPT-3 was roughly 100 times larger, yet it made more errors and more toxic output. Alignment, not sheer size, is what makes an assistant usable.
That’s also why competing assistants feel different despite sharing DNA. GPT, Gemini and Claude are all, at heart, decoder-only transformers pretrained on next-token prediction. The differences you notice come from training data, scale, and post-training choices, not a different core engine. Claude Opus 4.5 makes this concrete with a controllable “effort” setting. At medium effort it matches Claude Sonnet 4.5’s SWE-bench coding score while using 76% fewer output tokens. Engineering after pretraining now drives a lot of the personality gap between chatbots.
A caveat on the numbers, since you’ll see them quoted everywhere. Reports peg GPT-4 at around 1.8 trillion parameters in a mixture-of-experts design. That figure came from a leak, not OpenAI — treat it as an estimate. Want to see which labs lead on the models described here? We mapped the 2026 AGI development race separately.
Why chatbots hallucinate — and why it’s not a bug you can patch
Every heavy user has caught a model inventing a citation, a date, or a quote with total confidence. It’s tempting to call that a glitch. The research says otherwise.
OpenAI’s 2025 paper argues hallucination is mathematically unavoidable. Pretraining shows the model only fluent, positive examples of text, with no signal for what’s true and what isn’t. Even with perfectly clean data, the statistical objective still produces confident errors on facts that appear rarely. There’s no separate fact-checker inside to consult.
The incentives make it worse. Standard benchmarks score answers as right or wrong and award zero credit for “I don’t know.” Across thousands of questions, a model that always guesses confidently outscores one that flags uncertainty. We’ve trained these systems to bluff, then act surprised when they do.
Anthropic’s interpretability work adds a mechanism. In its models, refusal — “I don’t have enough information” — appears to be the default internal circuit. It’s switched on unless something overrides it. A hallucination happens when a “known answer” feature misfires and wrongly suppresses that refusal. The model then charges ahead as if it knows.
The key thing to hold onto: a hallucination isn’t the model lying. There’s no separate “make things up” mode to catch. The system is running the exact same token-prediction process that produces its correct answers. It simply can’t tell when that process has run out of reliable signal.
Do they understand language, or just model it?
Now the question everyone actually cares about. If a machine can plan a rhyme and chain facts, is it understanding — or performing a very good imitation?
The skeptical case has a name. In 2021, Emily Bender and co-authors coined “stochastic parrot” for a system stitching language together. It does so “according to probabilistic information about how they combine, but without any reference to meaning.” The phrase stuck. Sam Altman later mocked it on X with “i am a stochastic parrot, and so r u.” The jab suggested human thought might also be pattern completion all the way down.
Serious researchers land on opposite sides. Geoffrey Hinton argues you can’t reliably predict the next word at this scale without genuine understanding. The statistics force the model to build something that functions like it. Yann LeCun disagrees for structural reasons: a system trained on text alone “will never approximate human intelligence.” Words are too thin a signal to teach the grounded, causal picture of the world our senses give us.
Academics who’ve surveyed the fight refuse to call it. Melanie Mitchell and David Krakauer concluded in PNAS that there’s no expert consensus. The field, they argue, needs a real science of intelligence that can describe different modes of understanding. In 2026, Mitchell went further, coining “jagged intelligence.” It names how these models ace one problem and flunk a nearly identical one in bizarre, non-human ways. Whatever internal world model they hold, she argues, “is not like ours.”
So don’t let anyone sell you a verdict. This is a live, roughly even split among people who know the systems intimately. The mimicry-versus-meaning tension also runs through why AI deception matters more than passing the Turing test. Sounding human and being honest are not the same property.
Are large language models a step toward AGI, or an off-ramp?
This is where the mechanics collide with the biggest question in the field. If you keep scaling next-token prediction, do you eventually reach general intelligence — or hit a wall?
Start with a fixed point. DeepMind’s “Levels of AGI” framework grades systems on breadth and depth of capability. It classifies general chatbots like ChatGPT as “Emerging AGI” — matching or slightly beating an unskilled human across many tasks. That’s real, and it’s several rungs below where most people picture AGI.
The scaling camp points to emergent abilities: skills absent in small models that appear once models get large enough. Keep scaling, the argument goes, and new capabilities keep unlocking. But that evidence took a hit. Stanford researchers showed much of the apparent “emergence” is a measurement artifact. Pick a smoother metric and the jumps flatten into steady, predictable gains.
The people running these labs don’t even agree with themselves. In 2025, Altman said OpenAI is “confident we know how to build AGI as we have traditionally understood it.” Elsewhere he conceded that scaling alone is “no longer sufficient” and that new architecture is needed. Ilya Sutskever, once a scaling champion, now says 100x more pretraining compute won’t by itself produce a qualitative leap. LeCun is blunter still, calling text-only models a “dead end.” He’s betting on world models trained on video and sensory data instead.
Two credible camps, no winner. One says bigger transformers plus better training keep climbing the ladder. The other says LLMs are a useful component but structurally incomplete. Real generality, it argues, needs persistent memory, grounding, and causal reasoning that predicting text can’t supply. If you want the stakes of getting this right, we’ve asked whether AGI could become humanity’s last invention.
So how much should you trust what a chatbot tells you?
Trace the thread back and a practical rule falls out. The same mechanical process — predicting the statistically likely next token — produces both the genuinely useful answers and the confident fabrications. Truthfulness was never something pretraining directly optimized, so there’s no separate “hallucination mode” to switch off.
That mechanism tells you exactly where to lean in and where to double-check. Trust the output more when the answer is common and well-represented in training data. Think mainstream facts, broadly agreed knowledge, the shape of an argument. Trust it less when you need precise, rare, or highly specific recall: exact dates, real citations, niche statistics. Rare facts are precisely the ones most likely to trigger a confident guess.
There’s a fix on the table, and it isn’t more scale. Say leaderboards rewarded a calibrated “I don’t know” instead of scoring it zero. Then models would have a reason to flag uncertainty rather than bluff. That’s the realistic path — not eliminating hallucination, but making honesty pay.
Until then, the useful mindset is the one you’d bring to a brilliant, fast, occasionally reckless intern. Wonderful for a first draft. Never the final word. That caution matters even more as these models grow arms and legs. Once an LLM can take actions on your behalf, the trust question gets sharper. That’s exactly what we dig into when agentic AI acts and human control gets murky. Understand the machine, and you stop being surprised by it.
