Where AI Actually Helps, and Where It Just Costs Money

There is one shape of work this technology is genuinely good at. Lots of it, all broadly similar, with a right answer that exists and a cheap way to check whether you got it. Sorting incoming email into the queue it belongs in. Pulling the total and the reference off a supplier invoice. Answering the twenty questions your customers ask every week. Writing the first draft of something a person was going to edit anyway.

There is another shape where it costs money and returns very little. Decisions that come up rarely, carry real consequences, and where two experienced people in your own company would disagree about the right answer. Who to hire. Whether to extend credit to a new customer. What to say to somebody threatening to take you to court. You will get an answer for every one of those. Working out whether the answer is any good takes about as long as making the decision yourself.

Most of the money wasted on this went on the second shape dressed up as the first. Somebody picked the interesting problem instead of the frequent one. The demo went well, the pilot produced nothing anybody could point at, and by the next budget round the tool was still being paid for and no longer opened.

Volume comes first, because volume is the only thing that pays for the effort. A task somebody does twice a year is not worth automating even if the automation were flawless. You will spend longer building it, testing it and repairing it after the next process change than the task has ever cost you. Look at the jobs people do every day and have stopped noticing. They are invisible for exactly that reason.

Then ask how somebody would know it was wrong. If a person can check the output in a few seconds against something already in front of them, you are on solid ground. An extracted invoice total sits beside the invoice. A drafted reply gets read before it is sent. When checking takes as much effort as producing the thing yourself, the saving vanishes and you have added a step rather than removed one.

Then the cost of a mistake nobody catches. Mail filed in the wrong queue gets found and moved, and the damage stops there. A wrong figure written into your accounts travels. It gets summed, reported, and turns up months later in a meeting you did not want to be in. Same technology, same rate of error, completely different exposure.

Then ask whether the right answer stays right. Stable rules make a good candidate. A job where the correct answer changes every time your pricing, your policy or the law changes needs somebody keeping it current, and that job has no end date. Put that person in the budget before you start rather than after somebody notices the answers have gone stale.

Rare and consequential is the worst combination available. Rarity means you have too few examples to know whether the thing works. Consequence means finding out the hard way is expensive. Between them they remove both of your options, because you can neither test it properly nor afford to learn from live use.

Then anything where the real inputs were never written down. A great many business decisions rest on things that live in somebody's head. A conversation with the customer last spring. The fact that this supplier has been reliable for years and is having a bad quarter. A system reading only the records will produce a confident answer built from half the picture, and it will not tell you which half is missing.

Then anything you would have to explain afterwards. If a customer, an auditor or a court could ask why a particular decision went the way it did, you need a reason a person can stand behind. Generating a plausible sounding justification after the fact is a different thing from having decided for that reason.

And the jobs where nobody has agreed what good looks like. Ask two of your own senior people to do the same task separately, without conferring. If their answers differ in ways they would each defend, there is no right answer to aim at. Automating a judgement call mostly automates whichever opinion happened to be most common in the training material.

Most of the work worth doing sits between those two, in a pattern that produces a draft and waits for a person to approve it. The machine does the retrieval and the typing. The person keeps the decision, and with it the accountability.

Both of our own products are built that way, and not by accident. EnsureFix reads a ticket and opens a pull request, and a developer decides what merges. EzyConn answers a customer out of the documents you have already published, and passes the conversation to a person the moment it needs judgement, with the transcript attached so nobody has to start again. In both cases the difficult engineering lives in the handover rather than in the generation.

The gate only works if the person standing at it has attention to spare. Give one reviewer more drafts than they can genuinely read and approval turns into a formality, which is worse than having no review at all, because now there is a signature on the output. Size the flow to the reviewing capacity you actually have on a normal week, with people ill and other things urgent.

Watch for the drift as well. Review is careful at the start and casual later, because most of what comes past really is fine. Sample it. Have somebody check a slice of what was approved after the fact, and tell the reviewers that this happens.

The cost model is unfamiliar to most finance teams, because there is no licence to buy. You pay per request, and the request grows every time somebody adds another document to the context or another step to the chain. A feature that looks cheap in a pilot with a handful of careful users behaves differently once the whole company has it on a Monday morning.

Then there is the work around the model, which is where the real money goes. Somebody keeps the source content correct. Somebody maintains the set of test cases that tells you it still works after a change. Somebody handles everything the system escalates, and escalations are the hard cases by definition, so they take longer than the average case used to.

None of that argues against doing the work. It argues for putting those people in the plan at the beginning. A project that funds the build and forgets the upkeep gets worse quietly, and the decline usually gets blamed on the technology rather than on the staffing decision that caused it.

Before you commission anything, do this. Take a stack of real examples of the work from last month, including the awkward ones nobody likes, and write down the correct answer for each yourself. Do that part first, before you see any machine output. Deciding what counts as correct after you have seen the answer is how pilots fool the people running them.

Then put the same examples through whatever you are considering and compare. Count the outright errors. Count the answers that are technically correct and useless. Time how long the checking took you, because that figure decides whether the thing saves anything at all.

Two patterns in the result are worth noticing. If the errors cluster around one type of case, you can often route that type straight to a person and keep the rest. If the errors are scattered and unpredictable, no gate will catch them reliably, and what you are really looking at is full review of everything that comes out.

Start with the frequent, boring, low consequence thing that somebody in your company complains about. It will not impress anybody in a board meeting. It will be running within a month and it will keep running. Your team also learns how this behaves on your own material, and nobody can sell you that.

Leave alone whatever a competitor announced last quarter. You cannot see their running costs, their rate of error, or how many people they have quietly hired to check the output. Announcements are marketing artefacts. Judge the idea against your own work instead of against their press release.

And be willing to conclude that the answer is not a model at all. A surprising share of what arrives with us as an AI problem turns out to be a reporting problem, a form that asks for the wrong things, or two systems nobody ever connected. Those fixes cost less, they break less often, and nobody has to check their output every morning.