AI in Customer Support, What It Can and Cannot Do

It can answer questions you have already answered somewhere in writing. If the answer exists on your help pages, in a policy document, or in a PDF you send to every new customer, an assistant can find it and put it in front of the person asking, in whichever app they messaged you from. That is the honest boundary of the technology today.

It cannot decide anything. Whether to refund this customer. Whether this counts as an exception. Whether a complaint is about to become a legal matter. Those need a person who has the authority to say yes, and the design question worth spending real time on is how quickly and cleanly the conversation reaches that person.

We build EzyConn, which does this across web chat, WhatsApp, Teams, Slack and email. Most of what follows is what running it has taught us, including the parts that turned out to be harder than we expected when we started.

The mechanism explains most of the behaviour you will see, so it is worth understanding even if you never touch the configuration. The system searches your own material, finds the passages that look relevant to the question, and writes an answer out of them. Your documents are read at the moment the question arrives rather than memorised in advance.

Two things follow from that. The answer is only ever as current as the page it came from, so the policy you changed last week is the policy the customer hears about, provided somebody actually changed the page. And a question whose answer exists nowhere in your material will either be refused or invented, depending entirely on how the system was built.

The second behaviour is the one to test hardest before you switch anything on. Ask it things you have deliberately never published, ask them in the way a confused customer would, and watch what comes back. Do that before the vendor demo rather than after it.

A confident wrong answer costs you more than no answer at all. No answer produces a mildly annoyed customer who asks again. A wrong answer produces a customer who acts on it. They book the wrong appointment, expect a refund they are not going to get, or repeat it to somebody else as a thing your company told them.

Refusal therefore has to be built in on purpose. In practice that means requiring the answer to be grounded in retrieved material, checking that the material is genuinely about the question rather than merely worded similarly, and treating a weak match as a reason to hand over rather than a reason to try harder.

This is the part that takes the most tuning and it never really finishes. Set the bar too high and the assistant refuses questions it could easily have answered, which irritates customers and wastes the whole exercise. Set it too low and it starts filling gaps with plausible sentences. You find the line by reading transcripts, not by adjusting a setting once and walking away.

Write the refusal itself with some care. A message that says it cannot help and stops there feels like a locked door. One that says it does not have that answer, offers a person who does, and makes the handover in the same breath reads as service rather than as failure.

Everything useful about support automation depends on the handover being fast, obvious and complete. Fast means the customer does not spend four exchanges trying to escape. Obvious means there is always a way to ask for a person, offered in plain words rather than buried. Complete means whoever picks it up gets the whole transcript, so nobody is asked to explain their problem twice.

That last one is where goodwill evaporates. A customer who has already typed out the situation and is then asked to start again has learned something about how much you value their time. It is also the most common complaint about automated support anywhere, and it is an engineering choice rather than a limit of the technology.

Decide the handover triggers explicitly instead of leaving them all to a confidence score. Anything involving money. Anything where the customer has used the words complaint, cancel or legal. Anything asked twice, because a customer repeating themselves has not been helped. And anything at all from a customer who has asked for a human being.

Then staff the other side of it. A handover that lands in a queue nobody is watching at seven in the evening is worse than an assistant admitting it cannot help, because now the customer has been promised something and is waiting.

Once this is running, the quality of your support becomes the quality of your written material. That is a larger change than it sounds. Most companies have never had to treat help pages as production systems, and nothing in the way those pages were written assumed they would be read out loud to a customer by a machine.

The usual state of a help centre is a few well maintained pages, a larger set written years ago by somebody who has left, and at least one document that contradicts another. A human agent navigates that quietly, using judgement about which page is current. A retrieval system does not. It will find the old page, and it will find it repeatedly, partly because the old page is often better written than the one that replaced it.

So the maintenance job is real and it needs a named owner. Somebody removes superseded documents rather than leaving them lying around. Somebody marks which page is authoritative when two cover the same ground. Everything carries a date and an owner. The work is unglamorous and it returns more than any other part of the project.

Feed the failure log back into it. The questions the assistant could not answer are a list of the pages your customers wanted and you have never written. That list costs nothing to produce and is worth more than most customer research.

Draw the line at actions rather than at topics. Reading your published documents is safe. Reading a customer's account is a decision about identity and needs the same checks a phone agent would apply. Changing anything in that account is a different category again, and belongs behind a person or behind a very narrow, very well tested path.

Never let it make a promise it cannot verify. Delivery dates it has inferred. Compatibility it has assumed. An exception to a policy because the customer asked twice and firmly. Anything that commits your company should come out of a system of record or out of a human being with the authority to commit it.

Be careful with tone matching as well. An assistant that mirrors an angry customer produces a transcript that is unpleasant to read later, and support transcripts get read later far more often than people expect. By a manager, by the customer in a screenshot on social media, occasionally by somebody with regulatory authority.

The easy questions leave the queue first. What remains is denser: the confused, the angry, the genuinely complicated. Expect the time spent per ticket to rise even while the volume falls, and warn the team in advance. Without that warning they will read their own dashboard as evidence they have got worse at their jobs.

It changes what you should measure and what you should praise. Agents now spend the whole day on hard cases, which is more tiring than a mixed queue was. Acknowledge that out loud, because a team that feels the good part of the work was taken away will not help you improve the thing that took it.

It changes hiring too. The job becomes less about typing a familiar reply quickly and more about judgement, writing, and knowing when a policy should bend. Some of your fastest responders will not enjoy the new version. Ask them rather than assuming.

Give the team control of the assistant instead of making it something done to them. The people answering tickets know precisely which questions are being handled badly. If fixing a routing rule or correcting a source page means raising a request with a technical team and waiting, nothing ever gets fixed.

Start narrow. One channel, one clearly bounded set of topics, and a person reading every conversation for the first stretch. Reading transcripts is the only reliable way to learn what your customers actually ask, and it is never quite what your help pages assume they ask.

Say that it is automated. Customers work it out within a message or two anyway, and finding out after being misled costs you far more than the disclosure ever would. Being upfront also sets expectations sensibly, which makes people more forgiving of a refusal and quicker to ask for a person when they need one.

Then keep reading transcripts after launch, permanently, at a lower rate. Support assistants rarely fail dramatically. They drift into answering one common question slightly wrong and stay there for months, because the customers who were misled do not come back to tell you about it.