Agent vs Chat Assistant vs LLM Responder:

Why the Real Difference Is Who Decides the Next Step

A field guide to the three ways an LLM can sit inside a customer conversation — and the hybrid most enterprises actually end up building.

Everyone talks about “agents vs. LLMs” as if it’s a single line in the sand. In practice, there are at least three distinct flavors of LLM-powered systems, and a lot of enterprises build the middle one while believing they’ve built the third. The cleanest way to see the difference isn’t a definition — it’s watching all three handle the exact same problem.

The running example: a customer’s blender arrived broken, and they want a refund, through a company’s chat support.

Figure 1. Same customer message, three different systems — and three different answers to “who decides what happens next.”

1. The LLM as Responder

This is the simplest configuration: the LLM is a single-turn language layer. The customer describes the broken blender; the model produces one well-crafted reply — explains the policy, apologizes, maybe points them to an email address for a receipt. The moment that reply is generated, the LLM’s job is over. It doesn’t check an order system, doesn’t verify a warranty window, doesn’t issue anything. It has no memory of what happens after the message is sent. Whatever needs to happen next falls to a human, or to some separate piece of software entirely.

2. The Rules-Based Chat Assistant

This is where most production chatbots actually live, even when they’re described as “AI agents.” The LLM may interpret and generate — classifying intent, extracting fields, producing structured output — but the surrounding software still decides what happens next. A developer has pre-wired the decision tree: if the message mentions “broken” or “damaged,” trigger the return flow, ask for an order number, then a photo, then apply a hard-coded rule — orders under 30 days are auto-approved, everything else escalates to a human. The LLM can be doing real interpretive work at each step; it still cannot improvise a path nobody wrote into the tree.

3. The Agent

Get Somnath Biswas’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Here the LLM is handed a goal instead of a script: resolve this customer’s refund request. It’s given a toolkit — order lookup, a damage classifier, a refund processor, an escalation handler — and, at runtime, it decides the sequence itself. Which tool to call first. Whether to retry a failed lookup. Whether this case needs a path nobody explicitly coded, like realizing the order was a gift and looking up the giver’s account instead. If a step fails or the case falls outside policy, it doesn’t just stop — it adjusts, offers a partial credit, or escalates. In a well-built system it continues until it succeeds, reaches a defined stopping condition, or needs to hand off to a human.

4. The Practical Hybrid: Bounded Autonomy

Most real enterprise deployments don’t land cleanly on flavor two or flavor three — they land in between. Graph-based orchestration frameworks such as LangGraph illustrate this well. A developer still defines the overall map: the possible states and the allowed transitions between them, which sounds like the rules-based flavor. But the decision of which transition to take next, at each node, can be made by the agent’s own reasoning rather than a hard-coded condition.

The result is structure and guardrails from the graph, with adaptive judgment inside each node. Full autonomy where it earns its keep; hard boundaries where compliance or risk demands them.

Figure 2. The graph itself — which nodes exist, which transitions are possible — is developer-defined. Which transition actually gets taken at the decision node is chosen by the agent at runtime.

The sweet spot for most enterprises

Bounded autonomy tends to be where enterprise deployments actually converge: the agent gets to think and decide, but only within a map a human has already laid out. It’s rarely “pure rules” or “pure agent” in production — it’s a fixed skeleton with agentic joints.

5. A Second Axis: Agency vs. Authority

“Who decides the next step” is the key axis running through everything above — call that agency: how much discretion a system has over how it gets to an outcome. But enterprises need to answer a second, independent question too: authority — what the system is actually permitted to do once it has decided.

The two don’t move together. A highly agentic system can still have narrow authority: an agent may reason its way to “this refund is justified” with full discretion over how it got there, yet only be authorized to recommend that refund for a human to approve, not to execute it. Conversely, a fully deterministic, rules-based workflow can carry broad authority — every step predefined, but permitted to auto-issue refunds up to a set amount with no human in the loop at all. Agency is about how much the system is trusted to figure out; authority is about how much it’s trusted to act on. Mapping both, rather than just asking “is this agentic,” is what actually determines how much oversight a given system needs.

The Line That Actually Matters

The dividing line running through all of this isn’t whether an LLM is involved-every flavor uses one. It’s who decides the next step: a developer, in advance, writing every branch by hand-or the system itself, in the moment, reasoning a goal and a set of tools. For most real enterprise deployments, the honest answer is some of both, and the design question worth asking isn’t “should we use an agent,” but “which decisions, specifically, are we comfortable handing to runtime reasoning, which ones do we still want to write down ourselves- and, once that’s decided, what is this system actually allowed to do on its own?”