How model risk arises
A language model produces the most plausible continuation of what it has been given. Plausible is not the same as correct, and nothing in the output signals the difference — a fabricated statistic arrives in the same calm tone as a true one. Model risk is what that costs you once the output is acted on rather than read.
The exposure grows along two lines. One is reach: a wrong sentence in a private note costs nothing, the same sentence on a published page or in a reply to a customer is a claim your business has made. The other is autonomy: a model that suggests is easy to correct, an agent that sends, spends, edits or publishes on its own converts a wrong answer straight into an action. Risk sits where high reach meets low supervision.
Why model risk matters
Because the accountability does not move. If an assistant invents a specification, a delivery time, a legal condition or a refund policy and your team publishes it, the customer’s complaint and any regulator’s interest land on you, not on the tool. “The AI wrote it” is not a defence anyone accepts.
It also matters because the failures are unusually hard to spot. Bad code breaks loudly; a wrong figure in a report or a subtly incorrect claim in a page reads perfectly and can sit unnoticed for months, quietly informing decisions. And when a model is wrong at scale, it is often wrong the same way every time, so a single flawed instruction can propagate through a great many outputs before anyone notices.
Common mistakes with model risk
The first is treating it as a single dial to turn up or down. Drafting an internal outline and answering a customer’s question about pricing carry completely different consequences, and the same review requirement for both means people either waste time or start skipping it.
The second is trusting citations. A model can produce a source that does not exist, or one that exists and does not say what is claimed. If a link has not been opened, it has not been checked. The third is silent automation: workflows built so nobody can later reconstruct what the model was asked, what it returned, or which version produced the output that went out.
What to do about it
Sort tasks by what a wrong answer would actually cost. Low-stakes, reversible, internal work can run with light review. Anything customer-facing, anything that spends money, and anything making a factual or legal claim needs a person who approves it and is named as having done so. That approval step is the whole control.
Verify facts at the source rather than asking the model to confirm itself — a second opinion from the same system is not evidence. Keep prompts, outputs and approvals logged so a bad result can be traced and the instruction behind it fixed. Where agents act on their own, constrain what they are allowed to touch before you widen it. Sensible guardrails and a genuine human-in-the-loop step cost less than the retraction they prevent.