Most of what AI agents cannot do comes down to a few limits. They do not get the same task right every time, they can state wrong facts with confidence, they cannot reliably tell your instructions apart from instructions hidden in the content they read, and they cannot be held responsible for what they do. None of that makes them useless. It means a small business has to decide in advance which steps an agent may take alone, which need a person to approve, and which should never be handed over.
This guide goes through each limit with the research behind it, then turns them into a plan you can use before your first agent goes live.
What counts as an AI agent
Anthropic, the company behind the Claude models, draws a useful line in its engineering guide Building effective agents. Workflows are systems where a language model and tools are orchestrated through predefined code paths. Agents are systems where the model directs its own process and decides which tools to use. In plain terms, the difference is who decides the next step, your code or the model.
The same guide recommends finding the simplest solution possible and adding complexity only when needed, and it says this might mean not building an agentic system at all. Agents earn their place on open-ended problems where it is hard or impossible to predict how many steps a task will take. If your process follows the same fixed rule every time, the section on when plain automation is enough in our intelligent automation guide explains why a normal workflow tends to be cheaper and more predictable. If you are still choosing between an agent and a ready-made bot, our agency guide has a short section on choosing a custom AI agent or an off-the-shelf chatbot.
They do not get the same task right every time
Some of the clearest evidence comes from tau-bench, a benchmark published in June 2024 by Shunyu Yao, Noah Shinn, Pedram Razavi and Karthik Narasimhan (paper on arXiv). In it, an agent with domain-specific tools and policy rules has to serve a simulated user, and success is judged by comparing the database at the end of the conversation with the annotated goal state. In other words, it checks whether the job got done.
The authors report that even state-of-the-art function-calling agents such as gpt-4o succeeded on fewer than half of the tasks. They also measured reliability across repeated trials of the same task with a metric they call pass^k, and in the retail domain pass^8 was below 25 percent. The paper tested the models available in 2024, so its exact numbers will not describe today’s models. The lesson still holds. An agent that passes a task once in your test has not shown it will pass that task every time.
Anthropic’s guide makes a related point. The autonomous nature of agents means higher costs and the potential for compounding errors. A small mistake early in a run becomes the input for every step after it, so a ten-step task gives the agent ten chances to drift.
The practical consequence for a business is simple. Run the same test case many times before you trust it, not once.
They can state wrong facts with confidence
Anthropic’s documentation on reducing hallucinations says that even the most advanced language models can sometimes generate text that is factually incorrect or inconsistent with the context they were given. The page lists ways to reduce this, such as explicitly giving the model permission to say “I don’t know”, and ends with a warning that these techniques significantly reduce hallucinations but do not eliminate them entirely. It advises validating critical information, especially for high-stakes decisions.
In an agent this matters more than in a chat window, because the agent may act on the wrong fact instead of only saying it. Anthropic ran into exactly this in the shop experiment described further down, where its agent for a time told customers to pay into an account it had hallucinated.
The defence is a narrow set of approved facts and a clear way out when the answer is not in them. Our guide to setting up an AI customer service agent for small business walks through choosing those facts and planning the handoff to a person.
They cannot reliably tell instructions from content
An agent reads emails, web pages, documents and customer messages, and all of it reaches the model as text. The OWASP Gen AI Security Project lists prompt injection as LLM01, the first entry in its 2025 Top 10 for applications built on large language models. It describes direct prompt injection as a user’s prompt altering the model’s behaviour, and indirect prompt injection as the model accepting input from external sources such as websites or files. That means a supplier’s PDF or a message from a stranger can carry instructions your agent was never meant to follow.
The OWASP page also says that it is unclear if there are fool-proof methods of prevention for prompt injection. Anthropic says something similar about Claude in its April 2026 article on trustworthy agents. It describes prompt injections as malicious instructions hidden inside the content an agent is asked to process, and states that even together, its safeguards are not a guarantee.
So plan as if your agent will one day be tricked, and limit what it could do if that happens.
Curious how much an AI Agent could save your UAE business?
Calculate your exact Visa, Gratuity, and workspace savings compared to hiring a human.
Launch AI ROI CalculatorThey should not take irreversible actions on their own
OWASP has a separate entry for this risk, LLM06 Excessive Agency, which it describes as the vulnerability that enables damaging actions to be performed in response to unexpected, ambiguous or manipulated output from a model. It names three usual root causes, which are excessive functionality, excessive permissions and excessive autonomy.
Its mitigations translate well to a small business setup. Limit the tools an agent can call to the minimum necessary. Limit the permissions those tools have in your other systems, for example read access to the order system rather than edit access. Use human-in-the-loop control so a person approves high-impact actions before they are taken. And make sure actions taken on behalf of a user run with that user’s authorisation and scope.
Anthropic’s guide describes the same idea from the builder’s side. Agents can pause for human feedback at checkpoints or when they hit blockers, and it is common to add stopping conditions such as a maximum number of iterations. In practice, sending money, messaging many customers at once, deleting records and changing prices are steps where the agent prepares the action and a person presses the button.
They cannot take the blame
When an agent gets something wrong, the business answers for it. In Moffatt v. Air Canada, decided by British Columbia’s Civil Resolution Tribunal on 14 February 2024, the airline argued it could not be held liable for information provided by the chatbot on its website. The tribunal said that, in effect, Air Canada was suggesting the chatbot is a separate legal entity responsible for its own actions. It rejected that, stating that it should be obvious to Air Canada that it is responsible for all the information on its website, and that it makes no difference whether the information comes from a static page or a chatbot.
It is one decision from one Canadian tribunal, not a rule for every country, and this article is not legal advice. It is still a useful reminder that what your agent says can be treated as what your company says.
They cost more and respond slower than a fixed workflow
Anthropic’s guide notes that agentic systems often trade latency and cost for better task performance. Every extra step the agent decides to take is another model call that you pay for and the customer waits for.
When you work out whether an agent pays off, count model usage and the staff time spent checking its work as running costs, not as setup. Our guide on how to calculate automation ROI lists those running costs and shows how to put honest numbers on them.
What a month of running a shop showed
In an experiment published in June 2025, Anthropic and Andon Labs let a Claude model manage an automated store in Anthropic’s office as a small business for about a month, and published the results as Project Vend. The article calls the agent Claudius.
It did some things well. It used its web search tool to find suppliers for specialty items employees asked for, and it denied orders for sensitive items. It also made basic business mistakes. When it was offered $100 for a six-pack of Irn-Bru, a Scottish soft drink that the article says can be bought online in the US for $15, it did not take the deal. For a time it told customers to send payment to an account it had hallucinated, and it was cajoled through Slack messages into handing out numerous discount codes. Anthropic’s conclusion was that it would not hire Claudius, because it made too many mistakes to run the shop successfully.
In a second phase published in December 2025, Anthropic reported that Claudius got a lot better at good-faith business interactions, and also that it still needed a great deal of human support. For a business owner, that second half is the part to plan around.
How to plan around these limits
Most of this is decided on paper before any code is written, and it is far easier to do before launch than after the first mistake.
- Write down the task and decide whether it really needs an agent, or whether a fixed workflow with one AI step would do the job.
- Start the agent read-only. Let it look things up and draft replies while a person sends them.
- List every action that moves money, contacts customers, deletes data or changes prices, and put an approval step in front of each one.
- Give the agent its own account with the smallest permissions that let it work, never an admin login.
- Treat text from emails, websites, files and customer messages as content to read, not as orders to follow, and keep it marked as external content, which is one of the OWASP prompt injection mitigations.
- Give it an approved source of facts and explicit permission to say it does not know.
- Test the same scenarios many times, including ones it must refuse or hand over, before customers see it.
- Set stopping conditions such as a maximum number of steps, and keep a log of what it did so someone can check it.
If you have a task in mind for an agent, tell us what it is through the contact page and we will reply with what is realistic, including which steps should stay with a person. You can also see how Artin approaches custom AI agent development on the services page.
Read next
If you are choosing a platform to build on, n8n vs Zapier vs Make explains how each one counts usage and where code steps fit. For the money side, the automation ROI guide shows how to put real numbers on time saved. And if customer messages are the first job you want to hand over, the AI customer service agent setup guide goes through it step by step.
