AI customer service: what to automate and what to keep human (2026).
How to build an AI support agent that resolves real questions instead of deflecting them, which conversations it should hand to people, and how to measure whether it's actually helping customers.
Most companies have met the bad version of AI customer service: a chatbot that can’t answer, won’t let you reach a person, and makes customers angrier than before. The good version looks different. It resolves the questions it can, from your real policies and systems, and passes the rest to a person quickly, with the whole conversation attached. Customers get answers at any hour, and your team spends its time on the problems that need judgement and care.
What an AI support agent should resolve
- Status questions: orders, deliveries, bookings, applications and tickets, looked up in your systems and answered directly.
- How-to and policy questions: answered from your help centre, product documentation and policies, with a link to the source.
- Simple changes: updating an address, rescheduling, resending a document or cancelling within policy, done in your systems, not just described.
- Information gathering: collecting the details, photos and documents a case needs before a person picks it up, so nobody has to ask twice.
What it should hand to a person
- Complaints and angry or distressed customers.
- Anything involving vulnerability, health, safety or money beyond set limits.
- Questions the help content doesn’t answer: the agent should say so, not guess.
- Any customer who asks for a person.
The handover is the part most systems get wrong. A good one is fast, keeps the customer in the same channel, and gives the person the full conversation and what the agent already checked. Our note on fallback chains covers how to design it.
The measure of an AI support agent isn’t how many conversations it keeps. It’s how many it actually resolves.
Answers that are right
An AI agent is only as good as what it answers from. Three things matter most:
- Grounding. Answers come from your help content, policies and live systems, retrieved at the time of the question, never from the model’s general knowledge. Our note on retrieval that knows when to say “I don’t know” explains why abstaining matters.
- Current content. Outdated help articles become wrong answers. Fixing the knowledge base is often the first job.
- Testing on real conversations. Hundreds of real past tickets, including the awkward ones, scored before every change (see evals before features).
Channels
Customers contact you on web chat, email, WhatsApp, social messages and the phone. The same agent core (knowledge, system access, rules and tests) can sit behind each, with an interface suited to the channel. Many businesses start with the channel that has the longest queue, often email or WhatsApp.
The rules
- Tell people it’s AI. Required for chatbots in the EU under Article 50 of the AI Act, in force since August 2026, and expected everywhere.
- Treat messages as untrusted input. Customers, and sometimes attackers, will try to make the agent do things it shouldn’t. Its permissions must be limited in code (see prompt injection is an input-validation problem).
- Protect personal data. Only the data a conversation needs goes into the model and the logs, handled under the data protection law of each market you serve.
- Regulated sectors have extra duties. For example, UK financial firms must meet the FCA’s Consumer Duty, including for vulnerable customers.
How to measure it
| Measure | Why |
|---|---|
| Resolution rate (confirmed, not just “conversation ended”) | The real measure of whether it helps |
| Customer satisfaction on AI-handled conversations | Should match or beat your human baseline |
| Time to handover and handover quality | A slow or empty handover undoes the good work |
| Repeat contacts within a week | Catches answers that seemed fine but weren’t |
| Answer accuracy on a reviewed sample | Catches drift as products and policies change |
Questions support leaders ask
Should we buy a support AI product or build?
Buy if your help desk platform’s built-in AI covers your questions well. Build when the agent has to act in your own systems, follow your specific rules, or serve channels and languages the product doesn’t.
Will it replace our support team?
It takes on the repetitive volume. Teams usually shift towards complex cases, quality review and improving the knowledge the agent answers from.
How fast can we launch?
A focused agent on one channel, answering from good help content, can go live in weeks, with a person reviewing its conversations at first.
Modulus Labs AI builds support agents that resolve questions from your own content and systems, and hand over cleanly to your team. Tell us about your support queue, and we’ll reply within one business day.