chivvy. Start a conversation
What we doPractice PortalWorkHow we workGuidesArticlesAboutFree toolContactStart a conversation
Guide · Strategy

What AI is genuinely good at in a small business — and what it isn't

A straight capability map, with the failure modes named. Useful if you are trying to work out which of the things you have been promised are real.

4 min readWritten by Chivvy

The gap between what AI can do and what it's sold as doing is wide enough to waste a lot of money in. This is an attempt at an honest map, from building this software rather than marketing it: where it works, where it fails, and how to tell which side of the line a given idea falls on.

One framing to start with, because it predicts nearly everything below. Language models are extraordinary at reading, summarising and drafting. They are unreliable at knowing facts and at arithmetic. So the useful pattern is always: let ordinary software fetch the facts and do the sums, then let the model turn the result into words — or turn your words into a request. When you see an AI project fail, it has usually asked the model to be the database or the calculator.

Where it genuinely works

Turning data into explanation

Give a model the actual rows behind a number and it will write the two sentences that explain the movement — which product drove it, which accounts stopped, whether the overdue balance is one customer or a general slide. This is the single highest-value use in a small business, because it's the difference between a report being read and being acted on. The facts come from the data; the model only does the prose.

Drafting the repetitive correspondence

Chase letters, reorder nudges, follow-ups, appointment reminders, "here's what we agreed" summaries. These are high-volume, low-creativity, and they get written badly or not at all when a human is busy. A model that knows the customer's history writes a better first draft than a tired person at 5pm — and a person still approves it before it goes.

Triage and categorisation

Sorting inbound enquiries, tagging expenses to codes, spotting which emails need a partner rather than an admin. Accuracy here is good enough to be useful precisely because a mistake is cheap and visible — a misfiled ticket gets moved, not disastrous.

Extraction from messy documents

Pulling line items off a supplier invoice or a delivery note. This used to need bespoke templates per supplier and broke whenever a layout changed. It now largely just works, which quietly removes a lot of keying.

Answering questions in plain English

"How did this product do against last summer?" is a question most people can't turn into a report and won't ask anyone else for. Being able to ask it, and get an answer from your own data, changes who in the business is able to look things up — usually the most underrated effect of the whole exercise.

Where it doesn't work — yet, or maybe ever

Being the source of truth

A model asked about your prices, stock or VAT rules from memory will produce something plausible. Plausible is worse than useless when it's wrong, because nobody checks it. Facts must come from your systems, retrieved at the moment of asking. This is the failure behind most "the AI made something up" stories.

Arithmetic at scale

They are notoriously shaky at long chains of calculation, and — more importantly — they're non-deterministic, so the same question can give slightly different totals twice. Anything that has to reconcile, foot up, or be audited belongs in code and in a database, not in a prompt.

Anything unsupervised that touches a customer or money

Not because the drafts are poor; because the downside is asymmetric. An unreviewed email to your biggest account, an automatic payment, an automatic refund — the occasional error costs more than all the time saved. Every automation we build drafts and waits. The right question isn't "can it do this alone?" but "what does the once-in-two-hundred mistake cost?"

Regulated filing and advice

Anything with a statutory format and a liability attached — tax filings, regulated advice — needs deterministic, auditable software and a qualified human. AI can prepare, summarise and check around the edges of these processes, and that's genuinely valuable. It shouldn't be the thing that submits.

Judgement with thin data

"Should we open a second site?" is not an AI question. It depends on things not in any of your systems: your appetite for risk, your family, what your best manager is planning to do next year. Beware anything that dresses up a judgement call as an output.

A test you can apply to any AI proposal

Four questions
  • Where do the facts come from? If the answer is "the model knows", walk away.
  • What happens when it's wrong? Cheap and visible, or expensive and silent? Only automate the first kind.
  • Who approves before it leaves the building? There should be a name.
  • Could ordinary software do this? Often yes — and then it should, because ordinary software is cheaper, faster and repeatable. Reserve AI for the genuinely language-shaped parts.

The pattern that separates the two lists

Everything above follows one rule, and it's worth stating plainly because it predicts almost every success and failure.

Let ordinary software fetch the facts and do the arithmetic. Let the model read and write.

When an AI project fails, it has nearly always asked the model to be the database or the calculator. When one works, the model is doing what it's genuinely extraordinary at — turning a pile of retrieved rows into two clear sentences, or turning a question in English into a request the software can answer.

This is why the plumbing matters more than the model. Two businesses using the identical model will get wildly different results depending on whether the facts reaching it are joined-up, current and consistently defined.

The four failure modes, named

Worth being able to recognise these, because each has a specific fix.

Confabulation. The model produces a plausible fact it was never given. The fix is retrieval: never let it answer from memory, always give it the rows and instruct it to say when it doesn't know.

Silent drift. It worked in testing and degrades quietly as your data changes — a new product category, a customer type that behaves differently. The fix is spot-checking on a schedule, and logging every output so you can look back.

Over-trust. The output carries an authority the same information wouldn't have on a whiteboard, so nobody questions it. The fix is showing the workings — which records the answer came from, so it can be checked in ten seconds.

Scope creep by enthusiasm. It works for one job, so it gets pointed at a job with an expensive silent failure mode. The fix is the blast-radius question, asked every time rather than once.

The logging rule

Log every prompt and every response, with a timestamp and the records used. It costs almost nothing and it's the only way to answer "why did it say that?" three weeks later.

It's also what makes improvement possible. Without a log you're guessing about quality; with one you can read fifty real outputs and see exactly where the pattern breaks.

What it costs to run

Worth knowing, because the numbers are unintuitive in both directions.

Per-request costs for the current generation of models are small — writing a paragraph of commentary from retrieved data is fractions of a penny. A weekly report for one business is negligible. This is why "AI is too expensive for us" is rarely true any more.

What does cost money is volume without discipline: sending far more context than needed on every request, or running something on every record when it only needs to run on exceptions. The design decision that controls the bill is what you send, not which model you pick.

The bigger cost is almost always the plumbing — joining the data and agreeing the definitions. That's a one-off, and it's the part that makes the AI layer worth having at all.

How to test a claim

When someone proposes an AI capability, four questions establish whether it's real. They work whether or not you're technical.

  1. Where do the facts come from? If the answer is "the model knows", stop.
  2. What happens when it's wrong? Cheap and visible, or expensive and silent? Only automate the first kind.
  3. Who approves before it leaves the building? There should be a name.
  4. Could ordinary software do this? Often yes — and then it should, because ordinary software is cheaper, faster and repeatable.

That fourth question is the one that saves the most money, and it's the one nobody selling AI will ask you.

The honest summary

AI in a small business is not an oracle and it's not a member of staff. It's a very fast, very literate assistant with no memory of your business, which is exactly why the plumbing around it matters more than the model you pick.

Used that way, the returns are real and unexciting: reports that get written, follow-ups that happen, questions that get answered by whoever has them rather than whoever can build a spreadsheet. Hours come back, and they come back in the mundane half of the week — which is the half nobody started a business to spend their time on.

Keep reading

More guides.

Related service

This is a thing we build.

One useful email

A new guide or article most weeks — the practical kind, no product-pushing. Unsubscribe any time.

Get started

Want this done for you rather than by you?

Everything in these guides is something we build. If you would rather it simply arrived in your inbox every Monday, that is the job.

Straight to Andy, who'd be the one building it. No obligation, no follow-up sequence.