You want to know whether AI chatbots for business are worth trusting with customers. They are, for routine questions, if you ground them in current information and keep a human handoff for anything involving money, policy or sensitive data.
Chatbots are one slice of the wider field of ai tools for business, and this guide covers what they do well and where they fail.
| Item | Value |
|---|---|
| Lexis+ AI answers that were accurate and grounded in a 2025 legal research study | 65% of queries |
| Same study, accuracy of other tools | 41% and 19% |
| Air Canada tribunal order, fare difference | C$650.88 |
| Air Canada tribunal order, interest and fees | C$36.14 interest and C$125 in fees |
| OWASP Top 10 for LLM Applications, ranking of prompt injection | LLM01 |
These figures come from a guide to chatbot hallucination in customer service. Studies and cases get updated, so check the source before you rely on them.
What AI chatbots do well
A September 2025 article from the Northeastern Retail Lumber Association describes AI chatbots handling customer service inquiries and troubleshooting technical issues. It lists speed, scalability and around-the-clock availability as the main benefits.
The same article groups the useful tasks into a few areas:
- Handling customer service inquiries
- Troubleshooting technical issues
- Engaging potential buyers and recommending products
- Identifying high-quality leads
Each of these works best when an answer can be checked against a trusted source, such as a current price list or policy page. When the chatbot has to guess, quality drops quickly, as the next sections show. If support is your main use case, the guide to ai tools for customer service covers that area in more depth.
Where chatbots go wrong
The same association article warns that these tools can generate inaccurate or biased responses, especially when they pull from outdated or poor-quality datasets. It adds that chatbots have fabricated medical and financial advice in some cases. It also says they may collect and process sensitive data without informed consent or full transparency.
Two error types matter most. Misrepresentations are false or misleading statements, such as incorrect product details, policy terms or service information. Hallucinations occur when a chatbot gives responses that sound plausible but are factually incorrect or entirely fabricated, according to the same article.
NIST uses the term confabulation for fictitious, incorrect or fabricated output that appears plausible, as a customer service guide explains. The cause is structural: a large language model is a pattern engine that predicts the next token that best fits the conversation. It does not know what is true unless you give it a reliable way to check through retrieval, tools, rules or human oversight.
Hallucinations also come in different forms, and each needs its own fix. Examples from customer support include:
- A policy hallucination, where the bot invents or mutates refund or cancellation policies
- A pricing hallucination, where it invents discounts, shipping rates or taxes
- An invented link to a policy page that does not exist
- A claim to have completed an action, such as canceling a subscription, that the bot cannot actually perform
The same guide notes that with policy errors the customer acts on a false policy, which creates legal exposure. Pricing errors lead to revenue loss and customer disputes.
Does retrieval fix the problem?
Retrieval-augmented generation, known as RAG, reduces hallucination risk but does not eliminate it. Policies change, promotions end, inventory fluctuates and shipping estimates shift. A model trained on older data fills those gaps with whatever looks statistically likely.
A 2025 study of leading legal research tools, which are heavily retrieval-based, still found substantial inaccuracy and hallucination. In that study Lexis+ AI gave answers that were accurate and grounded for 65% of queries, versus 41% and 19% for other tools, as the customer service guide reports. The lesson is not about law: even retrieval-heavy products err when retrieval, ranking and response validation are not disciplined.
Security adds another layer. OWASP’s Top 10 for LLM Applications lists prompt injection as LLM01, and the guide warns that an attacker who gets your bot to ignore rules or fabricate policy exceptions creates a customer-facing failure.
What real incidents show
Public cases make the risks concrete. Each one below is a neutral example drawn from the same guide.
Air Canada, February 2024
In February 2024, Air Canada was ordered to pay a customer who relied on chatbot guidance about a bereavement fare refund. The airline argued the chatbot was a separate legal entity, and the tribunal rejected that argument entirely. It ordered payment of C$650.88 as the fare difference, plus C$36.14 interest and C$125 in fees, according to the guide.
The guide quotes the ruling: It makes no difference whether the information comes from a static page or a chatbot. Its takeaway is that in many jurisdictions, customers and regulators will treat chatbot output as your company speaking.
DPD, January 2024
In January 2024, DPD disabled part of its AI chatbot after users got it to swear and criticize the company publicly. The guide says the brand impact is real and immediate even when the content is just words.
Eurostar, December 2025
In December 2025, reporting on findings from Pen Test Partners described vulnerabilities in Eurostar’s AI support chatbot. These included issues that could allow malicious prompts or HTML injection. Eurostar said customer data was not at risk and that mitigations were applied, per the guide.
Hangzhou Internet Court, early 2026
In a case discussed in early 2026, a user sued after a generative AI app produced an incorrect answer. The Hangzhou Internet Court dismissed the claim, emphasizing warnings and safeguards in its fault-based liability framework, the guide reports. Legal outcomes vary by jurisdiction, but courts look for reasonable controls, warnings and oversight.
How to roll out a chatbot with safeguards
The sources point to a layered approach rather than a single fix. These steps follow from the risks above:
- Limit the bot’s scope to questions with clear answers, and keep policy, pricing and account details tied to verified sources.
- Ground answers through retrieval from content you keep current, because policies and promotions change.
- Add rules and human oversight, and decide in advance when the bot should cite a source, ask a question or hand off to a human.
- Test for prompt injection, using the OWASP list as a starting checklist.
- Be transparent about data collection, and obtain informed consent where sensitive data is involved.
- Review real conversations regularly, and fix the content or rules behind every wrong answer.
The customer service guide frames the goal as a chatbot that is fast and helpful but also knows when to cite sources, ask questions or hand off to a human. When you compare vendors against that goal, the guide on how to choose ai software for business covers how to weigh the options.
Frequently asked questions
Is a company responsible for what its chatbot says?
Show answer
In February 2024 a tribunal rejected Air Canada’s argument that its chatbot was a separate legal entity, and the guide says customers and regulators in many jurisdictions treat chatbot output as the company speaking. A case discussed in early 2026 at the Hangzhou Internet Court ended differently because of warnings and safeguards. Outcomes vary by jurisdiction, and this is general information, not legal advice.
What is the difference between a misrepresentation and a hallucination?
Show answer
A misrepresentation is a false or misleading statement, such as incorrect product details or policy terms. A hallucination is a response that sounds plausible but is factually incorrect or entirely fabricated, according to the September 2025 association article.
Can retrieval stop a chatbot from making things up?
Show answer
It reduces the risk but does not eliminate it. A 2025 study of retrieval-based legal research tools still found substantial inaccuracy and hallucination, as the guide reports.
What should a chatbot do when it is unsure?
Show answer
The guide says a well-built bot knows when to cite sources, ask questions or hand off to a human. That handoff matters most for refunds, pricing and account-specific requests.
Do chatbots create privacy risks?
Show answer
They can. The association article says chatbots may collect and process sensitive data without informed consent or full transparency, which raises ethical and privacy concerns.
