Most people first wonder how AI works the same way: they type a question into a chatbot and watch it answer. The reply comes fast. It sounds sure of itself. Sometimes it’s brilliant, and sometimes it invents a book that doesn’t exist.
So what’s going on inside? The plain answer is shorter than the hype suggests. An AI system learns patterns from a huge pile of examples, then applies those patterns to things it hasn’t seen before. Nobody writes the rules by hand. The system finds them.
That one idea explains a lot. It explains why chatbots write fluently and why they sometimes bluff. It even explains how a brain implant can turn neural spikes into text. But it doesn’t explain everything, and the gaps matter as much as the mechanism.
How AI works in one paragraph
Here’s the short version. Engineers gather data: text, images, brain recordings. They build a model, which is a huge stack of adjustable numbers called weights. Then they show it examples, measure how wrong its guesses are, and nudge the weights so the next guess is a little less wrong. They repeat that billions of times. Once training ends, the frozen model handles new inputs by running them through those learned weights. That’s how AI works at its core. Learn from examples, then predict.
The TL;DR:
- Data in. The model only knows what its training data and later tuning taught it.
- Learning, not rules. Developers design the learning process, not the answers.
- Prediction out. Every output is a best guess, including the confident ones.
- Opaque middle. Even the builders can’t fully explain how a model reaches a given answer.
How AI works vs traditional software
Traditional software follows instructions. A programmer writes “if this, then that,” and the computer obeys. When the program misbehaves, you can read the code and find the line that caused it.
AI flips that. Take a spam filter. The old approach meant writing rules by hand: block emails with “free money” in the subject, block certain senders, and so on. Spammers could just change their wording. A learning-based filter works differently. You feed it thousands of emails that people have already sorted into “spam” and “not spam,” and it works out which patterns separate the two. Nobody tells it which links or phrases matter. It figures that out from the examples.
Anthropic, the lab behind Claude, put the contrast in one line: “AI models are trained, not directly programmed.” The company then admitted the uncomfortable consequence. Because nobody writes the logic, nobody fully understands how these models do most of what they do.
That’s the trade, and it’s the first big shift from normal code. You get systems that handle messy, real-world input far better than hand-written rules. But you lose the ability to point at a line and say “there, that’s why.” Keep that trade in mind, because it comes back again and again.
The five-stage loop behind how AI works
Most explainers describe the process as a loop. The names vary, but the stages look like this.
- Collect and clean data. Teams gather examples and filter out junk. Meta, for instance, says it used heuristic filters, deduplication and text classifiers to clean Llama 3’s training set. It also used Llama 2 to help generate training data for those quality classifiers. So AI already helps build the next AI.
- Train the model. The model guesses, gets scored and adjusts. This is the expensive part.
- Store what it learned in a neural network. The “knowledge” doesn’t sit in a database of facts. It lives in the pattern of weights across the network.
- Run inference. You type a prompt and the trained model produces an output. This happens every time anyone uses the tool.
- Improve through feedback. Developers tune the model further with human ratings, new data or fixes for known failures. Then they ship an updated version.
Each stage shapes the product you end up using. Bad data in stage one, for example, leaks into every answer in stage four.
Training vs inference
People mix these up constantly, so here’s the split.
| Training | Inference | |
|---|---|---|
| What happens | The model adjusts its weights by learning from examples | The model uses fixed weights to answer a new input |
| How often | Rarely: once per model version, plus occasional tuning | Every single time someone uses it |
| Cost profile | One huge up-front effort | A smaller cost, repeated on every query |
| Does the model learn? | Yes, that’s the point | No, a chat doesn’t rewrite the weights |
That last row surprises people. When you correct a chatbot mid-conversation, it can use your correction for the rest of that chat. But your conversation doesn’t change the model itself. Lasting changes only come when developers run another round of training.
Inside a neural network: weights, loss and backpropagation
The neural network is where the learning actually happens, so it’s worth a closer look. No maths required.
Picture a network as layers of simple units. Each unit takes in numbers, multiplies them by its weights, adds them up and passes a result to the next layer. The first layer receives the input, such as the pixels of a photo. The last layer produces the output, such as “cat: 92%.”
Training runs as a cycle with three steps.
- Forward pass. The input flows through every layer and comes out as a guess.
- Loss. A formula scores how wrong that guess was. A big loss means a bad guess.
- Backpropagation and update. The system works backward through the layers and calculates how much each weight contributed to the error. Then it nudges every weight slightly in the direction that would have shrunk that error.
Then it does it again. And again. Over millions of rounds, the weights settle into a configuration that makes good guesses. This cycle is the engine underneath how AI works, whether the model writes poems or reads brain signals.
What does “deep” mean in deep learning? Just many layers. Early layers tend to pick up simple features, and later ones combine them into more abstract ones. Nobody assigns those roles, though. They emerge from training, which is exactly why the inside is so hard to read.
AI vs machine learning vs deep learning: what’s the difference?
These three terms get used like synonyms. They aren’t. Think of nested circles.
- Artificial intelligence is the broad goal: machines doing tasks that seem to need intelligence. It includes old rule-based systems too.
- Machine learning is the subset where systems learn from data instead of following hand-written rules.
- Deep learning is the subset of machine learning that uses neural networks with many layers. Today’s chatbots and brain decoders sit here.
So every deep-learning system counts as machine learning, and every machine-learning system counts as AI. The reverse isn’t true. A chess program built from hand-coded rules is AI, but it isn’t machine learning.
Three ways machines learn
Machine learning also splits by how the model gets its feedback.
- Supervised learning. Every example comes with the right answer attached, like those sorted spam emails. The model learns to match inputs to labels.
- Unsupervised learning. No labels. The model looks for structure on its own, such as grouping similar customers.
- Reinforcement learning. The model acts, gets a reward or penalty, and learns which actions pay off. Keep this one in mind, because it’s part of how chatbots get polished.
Large language models blur these lines. Their pretraining uses the text itself as the answer key: hide the next word, ask the model to guess it, then check. After that, they get reinforcement-style tuning on top.
How AI works in chatbots and large language models
Now for the part most people actually care about. When you ask a chatbot something, what happens?
Tokens and next-word prediction
First, the system chops your text into tokens. A token is a chunk of text, sometimes a whole word and sometimes part of one. The model turns each token into numbers it can process.
Then it does one thing. It predicts the most likely next token, adds it to the text, and predicts the next one. Word by word, the answer appears. The InstructGPT paper from OpenAI researchers states it plainly: the standard training objective for a language model is predicting the next token on a webpage.
That sounds far too simple to produce an essay. But scale changes things. Predict the next word across an enormous amount of text, with billions of weights, and getting it right starts to require grammar, facts, style and a fair amount of reasoning-like behaviour. That’s the engine inside every chatbot you’ve used.
Transformers and attention
Modern chatbots build on the Transformer architecture, introduced in the 2017 paper “Attention Is All You Need.” Its authors built it “based solely on attention mechanisms,” dropping the recurrence that older language models relied on. Attention lets the model weigh every token against every other token. So it can link “it” back to the right noun several sentences earlier.
The practical win was speed. Transformers ran in parallel and needed far less training time. The original model reached its translation results after 3.5 days of training on eight GPUs, which looks tiny next to what labs run today.
Human feedback: turning a predictor into an assistant
The InstructGPT team spelled out the catch. Predicting the next word on a webpage isn’t the same goal as following your instructions helpfully and safely. A raw predictor isn’t trained to help you. It’s trained to continue text plausibly.
So labs add a tuning stage. In the InstructGPT recipe, human labellers ranked model outputs, and those rankings guided further fine-tuning with reinforcement learning. OpenAI called the core technique RLHF, which “uses human preferences as a reward signal.”
The results surprised a lot of people. In human evaluations, raters preferred answers from a 1.3-billion-parameter InstructGPT over the 175-billion-parameter GPT-3. That’s 100 times fewer parameters. The same paper admits InstructGPT still made simple mistakes. Better behaviour without full reliability still describes chatbots today. If you want to see how RLHF stacks up against newer tuning methods, our comparison of RLHF vs Constitutional AI vs RLAIF lays out the differences.
The scale reality-check
How much data are we talking about? Meta says it pretrained Llama 3 on over 15 trillion tokens from publicly available sources. That’s seven times the data behind Llama 2, with four times more code. IEEE Spectrum reported the same figure independently and noted that Llama 2 trained on 2 trillion tokens.
What about GPT-4? OpenAI hasn’t disclosed its token count. Outside estimates put it around 13 trillion, but that’s an estimate, not a fact. You’ll also see confident dollar figures for training runs floating around online. Treat most of them as guesses, because labs rarely publish them.
How AI works when it “thinks longer”: reasoning models and agents
Generic explainers often stop at the chatbot. But how AI works in 2026 also depends on what happens after training. Two newer developments changed how frontier systems behave, and both kick in at the point of use.
Reasoning models and test-time compute
A reasoning model spends extra computation at inference time. Instead of answering at once, it works through intermediate steps, tries several approaches or checks its own work. Researchers call this test-time compute. One large 2025 study described it as the “dynamic allocation of compute during inference.” That study ran more than thirty billion generated tokens across eight open-source models, from 7 billion to 235 billion parameters.
Why does it matter? Because model size is no longer the only lever. Snell and colleagues found in 2024 that a compute-optimal test-time strategy beat a best-of-N baseline while using about four times less computation. They also pitted a smaller model with extra thinking time against a 14-times-larger model, with total compute matched. On easy and intermediate questions, the smaller model could match or beat the big one. The benefit depended on how hard the question was.
But more thinking isn’t free, and it isn’t always better. A 2025 paper found that longer chains of thought can impair reasoning in some domains. Then a 2026 study found diminishing returns at higher reasoning budgets. It also found that “overthinking” can lead models to abandon answers they’d already gotten right. So when a product brags about “thinking longer,” the honest follow-up is: longer on what kind of problem?
AI agents: models that act, not just answer
Agents are the other big shift. Broadly, the term covers systems built around a language model that can take actions toward a goal instead of only replying in a chat window. How they’re wired varies a lot from product to product, and the marketing runs well ahead of the evidence.
That’s also where trust questions sharpen. A chatbot that bluffs gives you a wrong answer. A system that acts on a bluff can do the wrong thing. Whether today’s long-running agents add up to something bigger is a live argument, and we dig into it in are long-horizon AI agents functionally AGI?
How AI works beyond text: images, robots and brains
The learning recipe doesn’t care whether the data is words. Swap the data and the output, and you get very different machines. The basic logic stays the same.
Images and video. Image generators also learn from huge sets of examples. The core idea holds: learn the patterns in the data, then produce new examples that fit them.
Robots. Embodied AI is the version that has to deal with the physical world, and the stakes change when the output is a movement instead of a sentence. It’s a big enough topic to deserve its own explainer, so we’ll leave it as a signpost here.
Brain decoding: AI that reads neural signals
This is where the “same recipe” idea gets genuinely remarkable. Strip how AI works down to its core, learning from paired examples and then predicting, and it applies to neural signals too. Brain-computer interfaces use machine learning to turn neural activity into text, speech or cursor movement. Researchers record brain signals while a person attempts to speak, then train a model to map those signals to words.
In 2023, a team published a high-performance speech neuroprosthesis in Nature that decoded attempted speech at 62 words per minute. That was 3.4 times the previous record. It’s still well short of natural conversation, which runs at about 160 words per minute. The system read spiking activity from microelectrode arrays implanted in the cortex.
The data problem here is the reverse of chatbots. Llama 3 had trillions of tokens. A brain decoder has whatever recordings one patient can provide. A 2024 Nature Machine Intelligence paper points out that matched neural and speech data is scarce and high-dimensional. Its fix paired an ECoG decoder with a differentiable speech synthesizer.
Then there’s a twist no chatbot has. In a 2025 Communications Biology study of imagined-speech BCIs using EEG, continuous feedback about the decoded activity was necessary for learning to happen. The authors concluded that combining machine learning with human learning makes BCIs easier to control. So the model adapts to the brain, and the brain adapts to the model.
Progress keeps coming. A 2026 Nature Medicine paper reported long-term independent use of an implanted BCI for both brain-to-text speech and cursor control. In September 2026, a Nature Neuroscience paper described decoding speech and gestures at the same time from a single implant, to animate a virtual avatar. For the full signal-to-output chain, read our breakdown of how brain-computer interfaces work.
Does AI actually understand? What’s proven vs claimed
This question sits behind most arguments about AI. The evidence splits into three tiers.
What’s proven: labs train language models to predict the next token, then tune them with human feedback. That mechanism fully explains why they write fluently.
What interpretability research suggests: something richer happens inside that prediction. Anthropic used a technique called circuit tracing on Claude 3.5 Haiku. MIT Technology Review’s coverage quotes outside researchers who call it a nontrivial engineering feat. Anthropic reports that Claude, while writing one word at a time, sometimes plans ahead. In a poem, it picked the rhyming word before writing the line leading up to it. The company also reports shared internal circuitry for the same concepts across English, French and Chinese. And for mental arithmetic, it found parallel computational paths rather than plain memorization.
What’s still unproven: understanding in the human sense, and anything like consciousness. Planning ahead inside a network isn’t the same as knowing what a poem means. Also keep the source in view. This is a company studying its own product, and one of its researchers said the work had really just begun. Anthropic’s newer research describes an emergent “global workspace” in language models, holding internal thoughts that never show up in the output. That’s a big claim. It’s also one lab’s claim, not settled science.
So the mechanism of how AI works doesn’t settle the understanding debate. “It’s just predicting the next word” is true. It’s also incomplete.
Why hallucinations happen
A hallucination is a model confidently giving an answer that isn’t true. For years, the explanation stayed vague: “it makes things up.” In 2025, researchers from OpenAI and Georgia Tech offered a sharper one. They argue that models hallucinate because training and evaluation reward guessing over admitting uncertainty.
Georgia Tech professor Santosh Vempala, a co-author, laid out the logic. Pretraining teaches a model to copy the patterns in its data, not to check truth. If the model hasn’t seen every true statement, it will produce plausible false ones. Then, if later scoring gives no credit for “I don’t know,” the model learns that guessing pays. His summary: “if your classifier can’t tell fact from fiction, your generator will hallucinate.”
That reframes hallucination as an incentive problem, not just a glitch. It also means a model’s confidence tells you nothing about its accuracy.
The opacity problem
Remember “trained, not directly programmed”? Now comes the bill. Nobody can fully trace why a model gave a specific answer. Dario Amodei, Anthropic’s CEO, argues in an essay on interpretability that for some uses this is “literally a legal blocker.” His example is mortgage assessments, where decisions have to be explainable. His framing: “We can’t stop the bus, but we can steer it.” Read that as advocacy from someone helping build the bus. The point about explainability still stands.
Myths about how AI works, against the evidence
Plenty of what people believe about how AI works doesn’t survive contact with the research. Here are the common ones.
| Myth | Reality |
|---|---|
| “AI looks up answers in a database.” | Knowledge sits in learned weights, not a lookup table. Anthropic reports Claude’s arithmetic runs on parallel computational paths, not memorized answers. |
| “If it sounds confident, it’s correct.” | Hallucinations are confident wrong answers by definition, and training often rewards guessing. |
| “Bigger is always better.” | A 1.3B InstructGPT beat 175B GPT-3 in human preference tests, and smaller models with extra test-time compute can rival much larger ones on some questions. |
| “More thinking always helps.” | Studies from 2025 and 2026 found diminishing returns, plus “overthinking” that can flip right answers to wrong. |
| “The builders know exactly how it works.” | Anthropic itself says it doesn’t understand how its models do most of what they do. |
| “Chatting with it teaches it.” | Inference doesn’t change the weights. Only new training does. |
| “Brain-decoding AI reads your mind.” | Speech BCIs decode attempted speech after training on that person’s recordings. In one imagined-speech EEG study, the user also had to learn, and feedback was necessary for it. |
Where AI is today: narrow AI, AGI and what comes next
Every system in this post is still a specialist in some sense. A brain decoder trained on one person’s speech attempts can’t drive a robot. A chatbot can write code, but it can’t fold your laundry. That’s narrow AI, even when it’s very wide narrow AI.
Artificial general intelligence, or AGI, would be, roughly, a system that could match people across a wide range of tasks. Even that rough line is contested, and our explainer on what AGI is shows why experts still argue about the definition. Nobody knows yet whether today’s methods lead there. Reasoning models and agents push the edge outward. But the evidence above shows the core recipe still has limits. It bluffs, it can overthink, and it stays opaque.
So when a headline says a model “understands” or “is basically AGI,” check the claim against the mechanism. Is the evidence about behaviour on a test, or about what’s happening inside? Those aren’t the same claim.
Why knowing how AI works matters for trust
You don’t need to code to use AI well. But knowing how AI works changes what you trust.
Once you know a chatbot predicts likely text, you stop treating its confidence as evidence. Once you know training rewards guessing, you start asking for sources. And once you know nobody can fully see inside these models, the push for interpretability and regulation stops sounding like red tape. It starts sounding like the bare minimum for systems that assess mortgages or sit inside someone’s skull.
That’s the human-centric point. The same learning recipe that drafts your emails helped decode a person’s attempted speech at 62 words per minute. Both deserve the same hard question: what does this system actually do, and who stays in control when it gets things wrong?
Pick the stage of the pipeline that still feels fuzziest and follow it through one of the guides linked above. Which part still feels like magic to you?
