Turning the Loop into the Human

How I designed, deployed, and keep steering an agentic content pipeline

Every morning at 8 AM, a workflow I designed picks one row from a spreadsheet, researches the topic, writes a full article, makes its images, and publishes it to a WordPress blog. On most days, nobody touches it. The human is still in the loop, though. I just didn’t put the human at a checkpoint. I built the human into the loop itself.

That is the idea behind this article. The usual picture of “human in the loop” is a person at a gate, approving or rejecting what a machine produced. It works for ten items a week. At a hundred, the person becomes a rubber stamp, and the gate gives a feeling of control rather than control.

The alternative I ended up with is to turn the loop into the human. My judgment lives in the system as sheets I edit, rules I wrote after each failure, and gates that hand back only the cases that really need me. I no longer review every output. I edit the loop that produces them.

Below I walk through how the pipeline is planned, how it is deployed, the theory that supports a system like this, and what keeping the human in the loop looks like in practice. It is not where I started, and it is not the final version.

Where it started

The first version, in May 2026, was a straight chain: read a task, search Google, write a brief, write the article, make images, post it. Six months later, the live workflow has 124 nodes. Fewer than a quarter of them write anything. Most of the rest check, ground, route, repair, or hold.

That ratio is the real story. Each time the pipeline did something I would not have done, I didn’t just fix the article. I added a part to the system so it would not happen again. The comments in the code read like a diary of those lessons:

None of these rules came from a design document. They came from watching the system fail in specific ways, then writing that experience down in a form the system could follow.

How the design is planned

The pipeline is planned as a set of separate jobs, each with one responsibility, and the jobs never trust each other’s output without checking. An article passes through seven stages:

1. Intake. It reads the Pending Tasks sheet and keeps exactly one row: today’s date, a blank Status, not a guest post, and a slug that isn’t already on the site.
2. Understanding. An agent analyses the task into a structured config. My manual overrides from the sheet are then applied on top, so my explicit choices always win over the model’s guesses.
3. Research. Google results through Serper are summarised into what the top pages cover. A keyword tool adds search data. Internal links are pulled from a pillar and cluster map, and new clusters are registered back into it.
4. Grounding. The brief is built only from material the system can trace: my style rules (word choice, voice and tone, terminology), a sheet of verified product facts, a list of trusted domains with scores, and, for comparisons, product manuals pulled from Notion and condensed by an agent.
5. Brief, then writing. A brief composer produces a contract for the article. The pipeline routes by layout (directory, comparison, alternatives or default), and only then does the writer agent draft.
6. Enforcement. Code, not a model, checks the draft against the brief: facts, prices, links, calls to action, images, and whether the topic already exists on the site.
7. Publish or hold. A clean article is posted with its category, images and status written back to the sheet. A failing one gets a repair attempt; if it still fails, it is held with its reasons and comes to me.

Only two boxes belong to a person: the task sheet at the start and fixing the cause at the end. The dashed line is the loop that keeps the human in it.

A few principles run through all of it:

• Plan before you write. The brief is a contract. The writer fills it; it doesn’t decide what the article claims.
• Separate the generator from the verifier. Models write. Deterministic code judges. A model never grades its own homework.
• One unit of work per run. It is slower, but each article’s context stays clean and traceable.
• Fail closed. When a check is unsure, the article is held, not published. A missing manual pauses a comparison and posts an alert to Slack instead of letting the writer guess.
• Every model has a backup. The analysis and writing steps each have a fallback model from a different provider.

How it is deployed

It runs on a self-hosted n8n instance, once a day, with spreadsheets as the control panel. I chose boring tools on purpose: each one is something a non-engineer on the team can open and understand.

Versioning is manual but deliberate. Before any large change I save the current state as its own inactive workflow: a frozen draft marked “DO NOT ACTIVATE”, a backup before the writer protections went in, and two “deployment candidates” that wait beside the live version, ready to swap in. Rolling back means switching which workflow is active.

The models change more often than the design. The node names in the editor still say things like “Claude Sonnet 4.6” while the model underneath has been swapped several times. That is a sign the design is right: the model is a replaceable part, and the structure around it does the real work.

The theory underneath

None of this is new thinking. I designed most of it by trial and error, and only later found that older fields had already named each idea. That is reassuring: it means the design rests on something sturdier than my taste.

Two of these carry most of the weight. Bainbridge explains why the approval gate fails. Ashby explains what replaces it: if one person can’t match the variety of the work, the person’s judgment has to become part of the system’s structure.

Why the approval gate fails

In 1983 Lisanne Bainbridge published a short paper called Ironies of Automation. It was about process plants and cockpits, but it reads as if it were written about AI pipelines. Her argument has four parts, and each one maps onto a person approving AI drafts.

The human gets the leftovers. Designers automate what they can and leave the operator what they couldn’t, which is usually the hardest part. In a content pipeline, the model does the fluent writing. The reviewer is left with the hard part: spotting the one wrong price in 2,000 confident words.

Monitoring is a task humans are bad at. Vigilance research going back to the 1940s shows that people’s attention drops when they watch for rare events. If 95 drafts in 100 are fine, the reviewer learns, correctly, that skimming is usually safe. That habit is what lets the 96th through.

Skills fade when they aren’t used. An editor who only approves stops writing, stops researching, and slowly loses the judgment the gate depends on. The better the automation, the faster this happens.

Get Atti Moradi’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

The takeover comes at the worst moment. When the system does fail, the person has to step in cold, without the context the system had, under pressure to clear the queue.

Large language models make this worse in one specific way: their errors don’t look like errors. A made-up statistic is written in the same calm, fluent tone as a real one. Research on automation bias (Parasuraman and Manzey, 2010) shows people tend to accept a machine’s output even when they have the information to reject it. Fluent prose strengthens that tendency.

Bainbridge didn’t conclude that humans should be removed. She concluded that the human’s role must be designed, not left over. Her own remedies map closely onto this pipeline:

What the theory says to build next

The theory explains why the current design works, and it also shows where the design is still weak. Five ideas point to the next steps.

1. Calibrated trust needs a measurement. Lee and See (2004) describe good reliance on automation as calibrated: trust that matches how reliable the system actually is. Management by exception has a blind spot. I only see what the checks catch, never what they miss. The fix is a small random audit: read one in every N published articles, even ones that passed every check. That gives a real miss rate and shows when trust is too high.

This is why I built a parallel system next to the pipeline: a content pipeline review dashboard where published articles are checked and scored independently of the checks that let them through. It is the measuring side of the loop, the place where trust gets calibrated against evidence. I will explain how it works in a coming piece.

2. Requisite variety can be counted. If my rules are meant to absorb the variety of failures, I should track that variety. Logging hold reasons by category shows which failures keep coming back (a missing rule) and which guards never fire (candidates for pruning). Without that log, the rule set only ever grows.

3. Autonomy should be earned per task type, not set once. Levels-of-automation research treats autonomy as a dial. Each article type can have its own setting. A layout with a low hold rate and a clean audit history earns more autonomy. A new or risky one, like comparisons with pricing claims, stays gated until it proves itself. The autonomy level becomes a result of the evidence, not a setting chosen up front.

4. Single-loop fixes are not enough; aim for double-loop learning. Argyris and Schön (1978) separate fixing an error inside the rules (single loop) from asking whether the rules and goals are right (double loop). Fixing a held article is single-loop. Changing the guard so the whole class of error disappears is double-loop. The next level is to ask, every so often, whether the pipeline is aiming at the right thing at all: the right topics, the right readers, the right measure of success.

Put together, the theory changes the human’s job description. It is no longer “approve the output”. It is “design what the system knows, measure how far to trust it, and decide what it should be trying to do.” Those are exactly the parts a machine can’t take over, because they define what the machine is for.

Turning the loop into the human

The human doesn’t sit at a gate in this pipeline. The human is spread through it, in four places.

1. Intent: the task sheet. I decide what gets written, on which day, under which keyword. I can override the model’s analysis field by field, and hand it the exact sources an editorial piece must use. To re-run an article, I clear one cell. The machine never chooses its own work.

2. Taste: the reference sheets. Word choice, voice and tone, terminology, verified product facts, and a list of trusted domains with a tier and a score. Only domains marked trusted with a score of 4 or more can be cited. These sheets are my editorial judgment in a form the pipeline can follow on every run.

3. Experience: the guards. Every guard is a past mistake I decided not to repeat. Check whether a topic already ranks on the site. Never state a price that isn’t in the trusted pricing source. Run one article at a time. Strip credentials from anything written to a sheet. Each one is a sentence of my experience, turned into code.

4. Judgment: the exceptions. When a check fails, the pipeline tries one repair. If the article still fails, it isn’t published. It goes to a Held Drafts register with the exact reasons, or, for a missing manual, to a Slack message. That is the only point where the system asks for me, and it asks with a specific question.

The loop that matters most is the one through me:

1. An article is held, with a reason.
2. I read the reason, not the whole article.
3. I decide if the rule was right, or if the system lacked something: a fact, a source, a style rule, a guard.
4. I change the system, usually a sheet row, sometimes a guard, so the whole class of problems is fixed.
5. The next run carries that judgment without me.

This is what I mean by turning the loop into the human. Reviewing outputs scales with the volume of work. Editing the loop scales with the variety of mistakes, and that number shrinks over time.

It has a cost. Encoded judgment piles up. My link and call-to-action validator alone is now about 100,000 characters of code. Rules written for last month’s failures can block next month’s good articles. So part of staying in the loop is pruning it: reading the rules again, merging them, and deleting the ones that no longer earn their place.

What comes next

The next version is about closing the outer loop: feeding what happens after publishing back into the rules. A weekly workflow already syncs results to a dashboard, and an engagement overlay is built but not switched on yet. The aim is for a held article and an article that underperforms to arrive at the same place, as a question to me about the system.

If you design AI systems, here is what I would pass on:

• Count your nodes by job. If most of your system generates and little of it checks, you have a demo, not a pipeline.
• Put your judgment where the system can read it. A sheet you edit beats an approval you click.
• Let models write and let code judge. Keep the verifier simpler than the generator.
• Make the system ask specific questions. “Held: price not in the trusted source” is something a person can act on. “Please review” is not.
• Treat every failure as a missing rule. Fix the class of problem, not just the article.
• Prune. Encoded judgment goes stale like any other code.

The human never left the loop. The job changed: from checking what the machine made to shaping the machine that makes it.

Further reading (cited from memory; check editions before publishing)

• Lisanne Bainbridge, Ironies of Automation, Automatica, 1983
• W. Ross Ashby, An Introduction to Cybernetics, 1956
• Stafford Beer, Brain of the Firm, 1972
• Raja Parasuraman and Dietrich Manzey, Complacency and Bias in Human Use of Automation, Human Factors, 2010
• John D. Lee and Katrina A. See, Trust in Automation: Designing for Appropriate Reliance, Human Factors, 2004
• Chris Argyris and Donald Schön, Organizational Learning, 1978
• James Reason, Human Error, 1990