Automation and AI

Model Risk

Also called AI risk

The exposure a business carries when a decision, a published claim or a customer reply rests on incorrect AI output.

Quick facts: Model Risk

Category
Automation and AI
Also called
AI risk
Level
Intermediate
Affects
Published claims, customer replies, budget decisions, legal exposure
Where to see it
Approval steps in n8n, Make or Zapier, audit logs, version history
In this article4
  1. How model risk arises
  2. Why model risk matters
  3. Common mistakes with model risk
  4. What to do about it

How model risk arises

A language model produces the most plausible continuation of what it has been given. Plausible is not the same as correct, and nothing in the output signals the difference — a fabricated statistic arrives in the same calm tone as a true one. Model risk is what that costs you once the output is acted on rather than read.

The exposure grows along two lines. One is reach: a wrong sentence in a private note costs nothing, the same sentence on a published page or in a reply to a customer is a claim your business has made. The other is autonomy: a model that suggests is easy to correct, an agent that sends, spends, edits or publishes on its own converts a wrong answer straight into an action. Risk sits where high reach meets low supervision.

Why model risk matters

Because the accountability does not move. If an assistant invents a specification, a delivery time, a legal condition or a refund policy and your team publishes it, the customer’s complaint and any regulator’s interest land on you, not on the tool. “The AI wrote it” is not a defence anyone accepts.

It also matters because the failures are unusually hard to spot. Bad code breaks loudly; a wrong figure in a report or a subtly incorrect claim in a page reads perfectly and can sit unnoticed for months, quietly informing decisions. And when a model is wrong at scale, it is often wrong the same way every time, so a single flawed instruction can propagate through a great many outputs before anyone notices.

Common mistakes with model risk

The first is treating it as a single dial to turn up or down. Drafting an internal outline and answering a customer’s question about pricing carry completely different consequences, and the same review requirement for both means people either waste time or start skipping it.

The second is trusting citations. A model can produce a source that does not exist, or one that exists and does not say what is claimed. If a link has not been opened, it has not been checked. The third is silent automation: workflows built so nobody can later reconstruct what the model was asked, what it returned, or which version produced the output that went out.

What to do about it

Sort tasks by what a wrong answer would actually cost. Low-stakes, reversible, internal work can run with light review. Anything customer-facing, anything that spends money, and anything making a factual or legal claim needs a person who approves it and is named as having done so. That approval step is the whole control.

Verify facts at the source rather than asking the model to confirm itself — a second opinion from the same system is not evidence. Keep prompts, outputs and approvals logged so a bad result can be traced and the instruction behind it fixed. Where agents act on their own, constrain what they are allowed to touch before you widen it. Sensible guardrails and a genuine human-in-the-loop step cost less than the retraction they prevent.

Do and do not

Do

  • Rank tasks by what a wrong answer would cost
  • Keep a named approver on anything customer-facing
  • Log the prompt, the output and the approval

Do not

  • Let an agent send, spend or publish unreviewed
  • Trust a citation you have not opened yourself
  • Read a confident tone as a checked answer

Questions people ask about this

Does using a better model remove the risk?

It reduces the frequency of errors, not the consequence of one. Stronger models are still confidently wrong sometimes, and a lower error rate can make things worse in practice, because reviewers relax exactly when the remaining mistakes are hardest to spot. Control the exposure with review and limits, not with model choice alone.

Where should we insist on human review?

Anywhere the output reaches a customer, commits money, or makes a factual, legal or medical claim. That covers published pages, ad copy, quotes and prices, replies to enquiries, and anything about safety or compliance. Internal drafts, summaries and brainstorming can run with much lighter checking, which is what keeps review sustainable.

How do we prove what an AI system did?

By logging it. Keep a record of the instruction given, the output returned, the version or tool used, who approved it and when. Without that trail you cannot tell whether a bad result came from the prompt, the source data or the model, and you cannot show a client or regulator how a decision was reached.

Related terms

Found this useful?

Share it, or ask an AI to summarise it

Back to the glossary

Knowing the term is the easy part

Applying it to your own site and budget is the work. Book a call and I will tell you what actually applies to you.