The Pipeline That Would Rather Withhold Than Delete a Live Page

01 A published page is a commitment, not inventory

A sibling piece on this journal (Truth-Gated Programmatic SEO) is about the gates that decide whether a candidate page ever gets written at all — most of them don't. That decision is cheap to get wrong and easy to reverse: a row sitting in blocked or queued costs nothing but a ranking window. This piece is about a different, harder decision: what the pipeline does when a truth-gate finding lands on a page that is already live, already indexed, already earning whatever ranking it earned. Pruning a candidate is bookkeeping. Deleting a live URL is not — and for one afternoon in August, the code that was supposed to know the difference didn't.

This repository is private, so what follows are receipts you cannot click — file paths, commit hashes, and a test file, quoted verbatim, that you're welcome to ask us to walk through live. Every fact below is checkable against this repo as of 2026-09-02.

02 Nine days dark, eighteen runs, one veto

The pipeline runs twice a day. On 2026-08-15, a commit landed with a blunt opening line: "Nothing has published since 2026-08-06. Eighteen consecutive scheduled runs failed and told nobody; the last two never started at all." Nine days, two runs a day — eighteen — and the commit log backs the number: the last successful publish before the outage was c61b025 ("auto-publish 2026-08-06 batch"); the next one was 985d8c5 on 2026-08-15, after the fix.

Three independent defects were each sufficient on their own to keep the pipeline dark, but the one this article follows is the first: check-pages.mjs collapsed ten heterogeneous audits into a single process exit, so one finding on the page a run had just generated also vetoed the deploy of everything else waiting behind it — including pages that had already passed their own gates and were just waiting to ship. The restart commit's own header comment names the actual page that jammed the pipeline for all eighteen runs: reviews/microsoft-clarity-review, which kept failing a record-consistency check because nextForTemplate() deterministically returns the first queued row, so the same failing row regenerated and failed identically every single run while three already-committed pages sat waiting on a deploy the same failure kept vetoing.

The fix was to scope blocking to what a run can actually withhold: a finding blocks only if it's attributable to a page this run would publish, or to no page at all in a section this run writes into, or has been promoted by a 7-day escalation clock. Everything else alarms instead of blocking. And the mechanism that makes "hard block" mean "hard-block the page, not the run" is a new script, scripts/pseo/withhold.mjs — its own header states the principle it exists to enforce: withhold the failing page, not the run.

03 The fix that could delete a live page

Commit 5bfbfe5 landed at 16:01 local time. At 18:42 the same day — about two hours and forty minutes later — a second commit landed: 93fa825, "close the holes the restart opened — no deleted live pages, no unattributable block, no rephrase-able exemption." Its message opens plainly: "Adversarial review of 5bfbfe5 found that the restart shipped four ways to lose content or ship an ungated page." The first and most serious of the four is the one this article is named after.

withhold.mjs's first version called rmSync on every path in the truth gate's withhold list, with no check for whether the path was actually tracked by git. Two ordinary, non-exotic paths could reach it holding a page that was already shipped: the 7-day escalation clock promoting a corpus finding (which carries the shipped page's file path), and plain regeneration, since scope.mjs puts modified tracked files in scope and two queued rows already had committed files that day. The commit message traces the consequence chain in full: the live page gets deleted; the deletion is counted as GENERATED content, so the commit step runs on what would otherwise be a quiet day; git add stages it; a leak assertion in the commit step catches the anomaly and aborts — which skips deploy, mark-deployed.mjs, and IndexNow, so the pending backlog doesn't ship either — and then a separate if: always() "corpus debt" step, which runs regardless of what happened upstream, commits the whole index and pushes a chore(pseo): corpus debt commit to main that removes a ranking page from the site.

That's a page deleted from production by a bookkeeping step that runs unconditionally, triggered by a truth-gate finding that was never supposed to have that authority in the first place.

04 The one question: is it in HEAD?

The fix is a single function, isTracked(), and everything downstream branches on its answer:

/**
 * Is this path in HEAD? The ONE question that decides delete-vs-restore.
 *
 * Fails CLOSED: if git cannot answer, the path is treated as tracked, i.e. it is restored rather
 * than removed. An unknown state must never resolve to "delete a file that might be live content".
 */
export function isTracked(rel) {
  try { git(['ls-files', '--error-unmatch', '--', rel], { stdio: ['ignore', 'pipe', 'ignore'] }); return true; }
  catch (e) {
    // exit 1 = git ran and said "not tracked". Anything else (git missing, not a repo, a lock) is
    // an answer we did not get, and we do not delete on a non-answer.
    return e?.status !== 1;
  }
}

Untracked paths — pages this run created and never committed — get rmSync'd, exactly as before; they should never reach the tree at all. Tracked paths take a different route entirely: git checkout HEAD -- <path>, not the two-dot checkout -- <path> it might look like at a glance. The distinction matters: checkout -- restores from the index, so a page that had already been git added would be "restored" to the very regenerated content that was supposed to be withheld, and would stay staged. Naming HEAD resets both the index and the worktree, so an already-staged page is un-staged rather than quietly kept. Deletions no longer count toward GENERATED either — a removal alone can no longer make the commit step think there's new content to ship — and the corpus-debt step now commits only its own pathspec, not the whole index, so it can never again sweep an unrelated file deletion into its push.

test/pseo/withhold.test.mjs replays five cases against a real git worktree of the repository — deliberately not a mock, because the defect was precisely that nobody had asked git the question. It proves an already-committed page survives byte-identical whether it's untouched, regenerated, or already staged before the gate runs; that an untracked page generated this run really is removed, because that's what withholding is for; and that a deletion never gets counted as generated content.

05 A finding with nowhere to land

The second hole in the same restart commit was quieter but shipped a page rather than losing one. The withhold list was built as blocking.map(f => f.file).filter(Boolean) — and nothing downstream ever checked whether any blocking findings had been silently dropped by that filter. A finding with no file attached — a queueable-evidence row queued with no SERP verdict, or a poisoned /build register — would be logged as blocking, withhold nothing, and let the page it named get committed, pushed, deployed, and marked published. The run only turned red at the very last step, after publication had already happened.

The fix, in scripts/pseo/gatepartition.mjs, changes what an unattributed blocking finding does: it withholds every scoped page in that finding's section rather than nothing, records the finding in a separate unattributed list, and — when there is genuinely nothing in scope to withhold in its place — sets a failClosed flag that makes the whole workflow refuse to commit anything at all. An unattributable finding used to pass quietly. Now it either takes a whole section down with it or stops the run outright; it never again ships the exact page it was complaining about.

06 An exemption that was a character count

The restart's second defect — a /build hub page whose mandatory sentence ("INSO is not quoting its own delivery cost on this page") was itself rejected by the undated-absence gate, making the whole template unable to emit — got a real fix: an exemption for claims whose subject is the house itself, since a first-person statement about what this site publishes can't go silently stale the way a claim about a vendor can. But the first version of that exemption, in scripts/pseo/buildgate.mjs, turned out to be a length budget rather than a grammar rule. Replayed against thirteen adversarial third-party sentences, seven took the exemption anyway — including "We do not publish Voiceflow's per-minute rate" and "We asked Ada and it publishes no per-seat price." The same sentence with one extra adverb tacked on flipped from passing at seventeen characters of tail to failing at twenty-nine, and only the failing half had ever been asserted in the test suite.

The replacement asks two questions instead of counting characters: is the house term the actual grammatical subject of the negated predicate (identity, not a prefix match — the old version let INSO-Bot read as INSO), and is the absent thing genuinely the house's own (a house term, or "its own" / "our own," inside the noun phrase itself)? Both have to be yes. Replayed against the same thirteen adversarial sentences, the bypass count is 0 of 13, and the one sentence that's supposed to pass — the mandatory /build hub line and four paraphrases of it — still does.

A fourth, smaller hole shipped the same day: the pre-generation step chained selftest && assets && cannibalization with no tolerance for a partial failure — the same shape as the original outage, a property of rows a run will never touch vetoing the deploy of pages that were already gated and only waiting on a deploy. It was split three ways: the self-test (which checks the gate code, not the corpus) stays fatal, while the two corpus-wide gates now only suspend new generation and let an already-gated backlog deploy regardless.

07 Two good pages, almost lost with the runner

Two days after the restart, a different failure mode showed up — not a truth-gate hole, but a plain git one, and it's the reason the pipeline's asymmetry (protect the live page, don't lose the good new ones either) needed one more fix. On 2026-08-17, a scheduled run generated four pages, withheld two on genuine truth-gate findings — a foreign-fact hit on build-vs-buy-ai-agent and a record-consistency hit on a comparison row, both visible in that day's queue diff as gate_fail_count: 1 entries — and committed the two clean ones: apps/alternatives/loox-alternatives and apps/reviews/rivo-review, 193 lines together, landing as commit 622326e. Then the run died:

[main 622326e] content(pseo): auto-publish 2026-08-17 batch
 2 files changed, 193 insertions(+)
error: cannot pull with rebase: You have unstaged changes.

The commit step is deliberately pathspec-scoped — it only commits the specific files the run means to publish — so generation's edits to queue.json, and anything withhold.mjs had just restored to HEAD, stayed unstaged in the worktree. git pull --rebase refuses to run against a dirty tree, so the push that would have shipped the two clean pages never happened, and both were lost with the runner — while the run's own output pointed at the withholding as the cause, which the fix commit (3dc2a39) is explicit it was not. The fix adds --autostash at all three rebase sites in the workflow: the dirty tree gets stashed for the rebase and reapplied after, so a partially-withheld run can still publish the pages that passed. Withheld files still never make it into the commit — that guarantee comes from the pathspec scoping, not from a clean tree — but a run no longer has to choose between protecting the two bad pages and shipping the two good ones.

08 Why the asymmetry is the whole point

As of this repo's current queue snapshot, 8,182 candidate rows are pruned, 715 are blocked, and 259 are queued — none of that is a loss, because none of it was ever a promise made to anyone outside this repository. A row can be re-scored, re-queued, merged into a sibling, or dropped entirely, and the only cost is a keyword's ranking window. A published page is a different kind of object: it's a URL that may already be indexed, linked, and ranking, and unpublishing it is a decision with external consequences that a truth-gate finding — which can itself be a false positive, as two of the three refusals documented in the sibling article turned out to be — shouldn't get to make unilaterally, in the middle of an unattended run, as a side effect of housekeeping.

So the standing rule, closed the same afternoon it was opened, is this: a truth-gate finding on a page this run is about to publish gets to withhold it — keep it off the site until it's fixed. A truth-gate finding on a page that's already live gets a report and a clock, in docs/seo/data/corpus-debt.json, that escalates to blocking if nobody looks at it for seven days. It never gets to reach for rm. Removing a page that's already earning a ranking is a decision that stays with a human, on purpose — the pipeline can refuse to add to the site all day long, but it doesn't get to decide, on its own, to take something away from it.

If you're evaluating whether an AI pipeline can be trusted with something that's already live — not just something about to go live — the question worth asking isn't only "does it have hard gates." It's whether removing something has the same guardrails as adding it, or fewer. That's the kind of system design we do.

Source table

07 Frequently asked questions

Did the pipeline ever actually delete a live, published page?

Not as far as this repository's commit history shows. The restart that introduced the bug (commit 5bfbfe5) landed at 16:01 local time on 2026-08-15; the fix (commit 93fa825) landed the same afternoon at 18:42, roughly two hours and forty minutes later. No auto-publish run appears in the git log between those two timestamps, and the outage meant the pipeline had not published anything in the days immediately before either. The bug was caught by adversarial review of the restart commit, not by a live incident.

What actually happens today when a truth-gate finding lands on a page that is already published?

The finding is reported, never silently acted on. If the page file is untracked (created this run, never committed), it is removed before anything can commit it. If the page file is tracked in git — meaning it already shipped — it is restored to its exact HEAD content, index and worktree both, and the finding is logged as a report against a 7-day escalation clock in docs/seo/data/corpus-debt.json. Nothing about a live page changes as a side effect of an automated run.

What is withhold.mjs?

scripts/pseo/withhold.mjs, the script that runs after the pSEO pipeline's truth gates produce a report. It decides which pages generated or touched this run get dropped from the commit set — "withheld" — because they failed a gate. Its central function, isTracked(), is the one check that decides whether a withheld path gets deleted or restored, and it fails closed: if git cannot answer, the path is treated as tracked.

How is this tested?

test/pseo/withhold.test.mjs runs five cases against a real git worktree of the repository — not a mock — because the defect this replaced was precisely that nobody had asked git whether a path was tracked. It asserts that an already-committed page survives byte-identical whether it is untouched, regenerated, or already staged; that an untracked page generated this run is actually removed; and that a deletion is never counted toward the run's GENERATED total.