Ask a chatbot to book a flight and it writes you a tidy paragraph about how to book a flight. Ask an AI agent, and the claim is that a ticket lands in your inbox. What is an AI agent, then, beyond a chatbot with better marketing? Mostly it’s plumbing.
The model underneath is often identical. What changed is the scaffolding around it: a loop that calls the model over and over, a set of tools it can trigger, and permission to keep going without checking in with you after every step.
That’s the whole trick. Small architectural change, enormous behavioral one. It also creates a failure mode that no amount of model progress fully removes, and that’s the part most explainers skip.
What is an AI agent, formally?
The word is about thirty years older than the chatbot era. Stuart Russell and Peter Norvig fixed the definition in their 1995 textbook: an agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment through effectors. They judged one by whether it maximizes its performance measure, given the evidence its percepts provide and whatever knowledge it starts with.
Nothing in that says “language model”. A thermostat fits. So does a chess program. Vendors picked up a thirty-year-old formalism and pointed it at their products, which is why the term feels precise and slippery at the same time.
The modern version narrows it. IBM defines an AI agent as a system that autonomously performs tasks by designing workflows with available tools, reaching past language into decision-making, problem-solving, and acting on external environments.
Anthropic’s engineering team draws the sharper line. They separate workflows, where predefined code paths orchestrate the model and its tools, from agents, where the model dynamically directs its own process and tool usage and keeps control of how the task gets done (Building Effective Agents).
Hold on to that split. Who decides the next step: the programmer, in advance, or the model, at runtime? Nearly every argument about what counts as an AI agent is that question in a costume.
Where a chatbot stops
A plain chatbot is one turn. Text goes in, the model predicts the next tokens, text comes back. Nothing persists inside the model between requests. Want a follow-up? You type again.
IBM’s term for these is nonagentic chatbots, and the list of what they lack is short: tools, memory, reasoning steps. They need continuous user input to respond. You are the loop.
That’s the real ceiling, and intelligence has little to do with it. A chatbot can’t decide to take a second action based on the result of the first, because there was no first action and no result. There was a reply. If you want the machinery under the prediction itself, our explainer on how large language models actually work covers it properly.
Now drop that same model into a loop that re-invokes it after every tool result. The ceiling lifts. Nobody retrained anything.
The loop: think, act, observe, repeat
The pattern most agents run has a paper behind it. ReAct, from Yao and colleagues at Princeton and Google in October 2022 and later published at ICLR 2023, proposed generating reasoning traces and task-specific actions in an interleaved way instead of choosing between them (ReAct). The model writes a thought, takes an action, reads what comes back, then writes the next thought with that result in hand.
Reasoning shapes the action. The action feeds real information back into the reasoning. Pure chain-of-thought reasons in a vacuum and drifts toward invention. Acting drags the model back to something external and checkable.
Here’s that loop with the flight example, one turn at a time.
Turn 1. Your request goes to the model alongside a list of tools it may call: searchflights, bookflight, sendemail. Each carries a name, a description, and a JSON schema for its parameters. The model replies with a thought and a call: searchflights(from: "BOM", to: "SIN", date: "2026-10-14").
Turn 2. The model ran nothing. Your code did. It executes the search, gets nine results back, and sends the whole conversation to the model again: your original request, the model’s thought, the call it made, and the nine flights as text.
Turn 3. The model reads the results, rules out two overnight layovers, and calls book_flight on the 07:40 direct.
Turn 4. The booking confirms. In it all goes again. The model writes a summary, calls send_email, and stops.
That run went fine. The loop earns its keep on the runs that don’t.
Say all nine results come back as overnight layovers. A static plan can’t help you there. It was drawn up before anyone knew what the search would return, and it says to book the best option, and there is no good option. The agent reads the nine results on Turn 3 before it commits to anything. So it writes a different thought. Widen the dates, drop the direct-only requirement, or call search_flights again on a nearby airport, then carry on from whatever that returns.
That is exactly what ReAct’s interleaving is for. Reasoning and acting alternate turn by turn, so every observation lands in front of the model before the next move gets chosen. A plan written up front can only be wrong quietly. A plan rewritten each turn collides with something real each turn.
None of which guarantees a save. The model can read nine layovers, shrug, and book one. But it can change course mid-task, and a fixed script simply cannot — that’s the capability the loop adds, and it’s worth more than any single tool in the list.
Look at what actually crossed the boundary. Structured JSON going out, text and JSON coming back. OpenAI’s own documentation is blunt about the division of labor: the model does not execute your function, it only decides which function should be called and what arguments to pass, and your backend does the actual execution.
Then there’s the detail that matters most later. Every turn re-sends the entire transcript. The model holds nothing between calls. Turn four isn’t the model remembering turns one through three. It’s turns one through three, typed out again, in full, as fresh input.
How an AI agent knows it’s finished
Starting a loop is easy. Stopping one is the interesting part, and that’s where the autonomy claim gets real.
On turn four above, the model produced no tool call. That absence is the stop signal. The harness sees plain text where a function call would sit, and exits. Nobody wrote a rule anywhere saying “book one flight, then quit”.
IBM frames the broader pattern in three stages: goal setting and planning, reasoning with the tools on hand, then a reflection step where the agent stores what it learned for future work. Anthropic’s definition reaches the same place from another angle. An agent keeps control over how it accomplishes a task, and that necessarily includes judging when the task is done.
Wrap a hard turn cap around it anyway. The model decides in principle. Whatever budget you set decides in practice.
How an AI agent picks a tool, and how it gets that wrong
Tool choice involves no magic. The developer writes each tool’s name, its description, and a schema for its arguments. The model reads those descriptions as text in its context and produces a tool name plus arguments that fit the schema. Text in, structured text out.
Which means tool descriptions are prompts. Careless ones break agents.
Anthropic’s engineering team went hunting for why production agents choose badly, and found the cause in tool design rather than model capability. When tools overlap in function or have a vague purpose, they wrote, agents get confused about which ones to use. The damage shows up four ways: the wrong tool, the right tool with wrong parameters, too few calls, or a tool response the model reads incorrectly.
Their example is almost funny. One ambiguous parameter description pushed Claude’s web-search tool into appending “2025” to every query it ran, quietly degrading results across the board. No model flaw involved. Just a sentence somebody wrote without thinking.
So the emerging standards work on how agents find and trust tools matters more than it sounds, which we dug into in Agentic Resource Discovery. Get the descriptions wrong at scale and the strongest model available still fumbles.
Agentic AI, generative AI, and the chatbot with function-calling
Ask three vendors where the line sits and you’ll get three lines, each drawn just past whatever they sell.
Try the honest version. Autonomy is a dial, not a switch, and the variable that matters is how many loop turns run before a human gets a say.
- Zero turns. A chatbot. You are the loop.
- One turn. A chatbot with function-calling. It looks up the weather, tells you, stops.
- Several turns, fixed order. A workflow. Code decides the sequence, the model fills the blanks.
- Several turns, model’s choice. An AI agent. It picks each next step and decides when to stop.
Anthropic’s workflow-versus-agent split lands across those last two bands, and they’re unusually honest about what follows. Their advice is to find the simplest solution possible and only increase complexity when needed. Most tasks never need an agent. One well-equipped model call beats a loop with four fresh chances to go wrong.
“Generative AI” sits on a different axis altogether. It describes what the model produces, not how much rope it gets. Our breakdown of narrow, generative and agentic AI untangles the three labels people keep swapping by accident.
The five types of AI agents, and why the list feels dated
Every explainer reaches for the same taxonomy, and it also traces back to Russell and Norvig.
- Simple reflex. Acts on the current percept alone. If smoke, then alarm.
- Model-based reflex. Keeps an internal picture of the world, so it can act on what it can’t currently see.
- Goal-based. Chooses actions by whether they move it toward a stated goal.
- Utility-based. Weighs outcomes by preference when several routes reach the same goal.
- Learning. Improves its own behavior from feedback over time.
Good lens. Poor map of what people actually ship. A modern LLM agent is usually a goal-based and utility-based hybrid, written as prose inside a prompt instead of a hand-coded rule table, and driven by the ReAct-style loop above. Nobody builds simple reflex agents out of frontier models. The taxonomy describes the shape of a decision, not the shape of a system.
Why AI agents fail: errors compound
Here’s the consequence of that re-sent transcript, and it’s the most useful idea in this post.
When a model gets something wrong in a single chat turn, you see it and correct it. When an agent gets step three wrong, step three goes into the transcript. Step four reads it as established fact. So does step five. The mistake stops being a mistake and becomes a premise.
Degradation doesn’t stay linear. It accelerates.
METR, a nonprofit that evaluates AI systems, put numbers on the other side of this. They measure the length of task an agent can complete with 50% probability. That length has doubled roughly every seven months across about six years of frontier models (Measuring AI Ability to Complete Long Tasks). At publication in March 2025, Claude 3.7 Sonnet’s measured horizon sat near one hour.
Two things live in that finding, and most coverage quotes only the first. Yes, the curve is steep. But ask why the metric needs “50% probability” written into its definition at all. Reliability falls away as tasks lengthen. The benchmark is built around that decay because the decay is the defining property of what it measures.
Context is the second wall. The transcript grows every single turn, so a long task eventually fills the window with its own history, crowding out or summarizing away the instructions that started it. Bigger context windows push the wall further out. They don’t remove it.
Who’s holding the leash?
Most writing about AI agents treats autonomy as a finish line. Not everyone agrees.
A 2025 position paper by Margaret Mitchell, Avijit Ghosh, Alexandra Sasha Luccioni and Giada Pistilli, researchers at Hugging Face, argues that fully autonomous agents should not be developed at all without governance around them. Their case rests on concrete categories: security and cybersecurity exposure once an agent acts without oversight, bias traveling through decisions nobody reviews, the loss of meaningful human control, and the deskilling that follows from delegating too much for too long.
You can disagree with their conclusion. The mechanism is harder to argue with. Each loop turn you remove a human from is one more place a compounding error passes unchecked. Autonomy and error-catching trade against each other directly, and that trade doesn’t improve as models get better. It gets more tempting.
We’ve written before about what happens to human control when agentic systems act on our behalf. The short version: the live design question isn’t how much autonomy you can grant. It’s where you put the checkpoints.
So how do you tell?
Next time someone sells you an AI agent, skip the demo and ask two questions. How many steps does it take before a human sees the output? And what happens on step four when step two was wrong?
The first answer tells you whether you’re looking at an agent or a chatbot in a nicer jacket. The second tells you whether anyone on the team has thought about the failure mode at all. Most product pages answer neither, which is itself informative.
“Agent” is doing a lot of work as a word right now, and a fair share of it is marketing work. The engineering underneath is refreshingly dull: a loop, some JSON, a transcript that never stops growing, and a judgment call about when to let a human back in. That last piece isn’t a technical problem. Which is exactly why it’s the one worth arguing about.
So go run those two questions past whatever system at your own workplace calls itself agentic, and see what survives.
