The check step: what goes between AI output and the person who gets it

The check step: what goes between AI output and the person who gets it

A check step is a rule that runs after a model produces something and before anyone acts on it. Most AI workflows have none, because the output arrives looking finished and finished-looking work does not invite inspection. Here are the three kinds worth building, where each belongs, and why the check should exist before the workflow it protects.

The reason to care is narrow and specific. AI failures are rarely loud. The output arrives on time, in the right format, in a reasonable tone, and one number in it is wrong. Nothing in that description triggers anyone to look.

What counts as a check step?#

Three kinds, in order of how little effort they take.

A range check. A rule about what an acceptable answer looks like, applied automatically. Revenue figures fall between these two numbers. The summary is between two hundred and four hundred words. Every row has a date. Anything outside the range gets held rather than sent.

Range checks catch the failures that are obviously wrong once stated, which is a much larger share than people expect.

A stated-uncertainty step. Asking the model, in a separate pass, what it was least sure about and what it had to assume. Not "check your work", which produces agreement. A specific request for the weakest part of what it just produced.

This works because the model has the information and is not asked for it by default. It fails when phrased as a yes or no question, because then you are asking it to agree with itself.

A spot check. A person reads three random outputs a week. Not the first three, not the ones that look interesting. Three chosen without looking.

This is the least sophisticated and catches the most, because it is the only one of the three that can find a failure nobody predicted.

Where does the check belong?#

Not at the end.

The instinct is to review the finished thing. The problem is that a finished thing invites approval. It has a structure, a conclusion, and the visual signals of completed work, and a reviewer looking at it is checking presentation rather than substance.

Put the check on the input to the risky step instead. Before the email sends, before the figures reach the deck, before the batch is written. At that point the reviewer is looking at something obviously provisional, and provisional things get read properly.

The practical version of this rule: approval belongs where the cost lands, not where the work ends.

When should the check exist?#

Before the workflow does.

This sounds backwards and is the single highest-value habit in this whole area. If a task is worth automating, spend the first week doing it by hand with the check in place. You learn three things you cannot learn any other way: whether the check is measuring the right thing, how often it should fire, and what the failures actually look like.

Then automate. The workflow arrives with a working check attached instead of a promised one.

The alternative, which is what usually happens, is that the automation ships, runs unwatched for two months, and the check gets added after the first visible failure. By then the check is designed around one incident rather than the pattern.

What does a check step cost?#

Less than people assume, and the comparison is not the one they make.

The comparison people make is check versus no check. The real comparison is fifteen minutes a week versus the afternoon spent reconstructing what went out and to whom.

Building one, in practice#

Take a workflow you already run. Answer three questions about it and the check writes itself.

1. What does a wrong answer look like here, specifically? Not "bad output". A number in the wrong range, a missing section, a name that should not appear.
2. Which of those could a rule catch without a person?
3. For the ones a rule cannot catch, who reads three of these a week?

If question one is hard to answer, that is the finding. A workflow whose failure mode you cannot describe is a workflow you cannot check, and that is worth knowing before it runs unattended.

I want to build a check step for a workflow I run. Work through this with me, one question at a time, and do not accept a vague answer.
1. Ask me to describe the workflow and what its output is used for.
2. Ask me what a wrong result would look like specifically. Push back if I say "bad" or "inaccurate" and make me name the actual failure.
3. Tell me which of those failures a simple rule could catch automatically, and write the rule.
4. Tell me which ones need a person, and how often they would need to look.
5. Tell me where in the workflow the check should sit, and why there rather than at the end.
End with the smallest version I could put in place this week.

The part this does not solve#

A check catches the failures you designed it to catch. It says nothing about the ones you did not think of, and a check that has never fired tells you nothing at all about whether it works.

That is a separate problem and it is the more interesting one.

Ahead pairs you with an expert operator who builds this into your own workflow, in your own tools. If you have a workflow running unattended and cannot say what would tell you it had gone wrong, that is a good first hour.