Designing AI workflows with explicit boundaries
My working method: define the decision, separate implementation from acceptance, and make uncertain outcomes visible. The examples below show which checks run automatically, which need an operator, and which remain judgement calls.
The aim is useful software that people can operate and question—not more process for its own sake. Checks reduce specific risks; they do not guarantee a useful product or correct reasoning.
Method notes: assumptions, historical failures & limits
The premises
The premises, stated so you can disagree with them:
-
A1A new model call can use only the context made available to it.
Conversation history, saved notes and memory tools can carry earlier work forward, but only when the workflow retrieves and supplies them. A convention stored somewhere is not necessarily present in the next decision.
-
A2AI can produce changes faster than I can validate them.
A draft can arrive quickly; understanding its effects, testing its boundaries and deciding whether it solves the right problem still take work. Faster generation does not remove that work.
-
A3Errors have asymmetric costs: making one can be cheap, finding it expensive, and fixing it after other work depends on it more expensive still.
The three costs are not merely different sizes. They point in different directions, so any process tuned for the cheap one is tuned against the expensive one.
What follows
What follows, and what each one forces:
-
T1A check is only as independent as the boundary protecting it. from A1
A model’s self-review is not independent acceptance evidence. I also use rule engines, process exit codes and separate implementations. Their results still depend on the checks and configuration: a recorded probe disabled a guard by adding a trailing comment to the line it read, while it continued reporting zero violations.
-
T2Faster generation makes verification more important. from A2
Building a useful product still matters. Alongside implementation, I invest in defining what counts as done, checking the result, and testing whether the checks can actually reject a broken example.
-
T3Checkpoints have to be closer together than the rate at which errors get built on. from A3
Cheap checks move closer to the tool call so mistakes are caught before more work depends on them. But hooks see only the operations they inspect: two shell one-liners went through the same hook and the files were really written. That channel is checked with before-and-after snapshots instead—detection, not prevention.
-
T4Intent needs an execution path, not only a place to be written. from A1
A rule in a document does not run itself. Where possible, I pair it with an executable check. Programs and configuration can also be changed or bypassed, so the check’s own inputs and execution need review.
-
T5Repeated failures without a visible signal can accumulate unnoticed. from A3
Silence is not a signal, so nothing throttles it. A cleanup routine captured output without checking the return code. The recorded incident accumulated 2,983 leaked worktrees (about 26 GB), plus about 60 GB of pytest temporary files: roughly 86 GB in total. Failures now have to be visible even when nobody is watching.
What it does not solve
What it does not solve — measured, not anticipated. An earlier version of this list was written before the work started and read better; these are the ones that turned up afterwards:
-
A check can tell you a claim does not match the facts. It cannot tell you the reasoning behind the claim was wrong. The claim stays well-formed, the check has no opinion about it, and the mistake is somewhere the check cannot see.
COSTOne bad estimate of a root cause sat behind a green board for twenty-six commits. What broke the deadlock was someone going and measuring it, which is not a thing a gate can be made to do. This is the limit, not a gap: the honest response is to say which conclusions rest on judgement and stop implying the machinery covers them.
-
The coverage of a check is not observable from its result. Where a check finds its subject by matching text, "matched nothing" and "there is nothing to match" arrive as the same answer — and it reports a pass either way.
COSTEight checks audited one by one; six had the same defect, every one reproduced by hand rather than reasoned about. Changing two words in a comment made an over-budget document quietly compliant. It is worse than a silent failure, because a silent failure is silence and this is a green tick.
-
A check whose switch is reachable by the thing it checks. Adding a trailing comment to one line of a protected-paths list made the pattern match nothing at all — so a file could be edited and the guard watching that exact file reported zero violations. One edit did the trespass and hid it.
COSTOpen in the cited historical record; current resolution is not asserted here. The proposed fix needs every pattern to match an existing target, and intentionally empty patterns to be explicit. Half a fix could restore confidence without restoring coverage.
-
A historical nonconstant audit on 22 August 2026 counted twenty-nine protocol rules: five classified as mechanised, eleven as implementable gaps, twelve as judgement boundaries and one permission. That classification even included a human-checked requirement. It is a record of that phase, not a count of today’s installed gates or a claim that every labelled control runs automatically.
COSTPublishing the classification exposes how much still relied on judgement or unfinished work. The later README records five of thirty-three protocol rules at its own dated measurement point; the protocol and the ten registered gate scripts are different inventories.
-
Hardening the checks is optimising for a user who does not exist yet, and there is no natural stopping point — a checking system can always be made more robust.
COSTSeven consecutive working sessions went entirely into meta-work and moved the one thing that would produce a user precisely nowhere. There was a written threshold for this: meta-work above fifteen per cent triggers a cleanup. It ran at roughly a hundred, and nothing was checking the threshold. The rule against unchecked rules had no check.
These arguments grew from the working notebook now published with nonconstant, with examples from my other projects. Agents helped write those notes; their first-person accounts are not automatically my own observations. My contribution is the decisions, acceptance criteria and review, including rejecting earlier conclusions. Historical figures refer to the recorded phases, not current totals. Public source makes a claim inspectable; it does not by itself prove that the claim is correct.
PUBLIC CHECK & LOCAL RELEASE CHECKS
The public rules digest can be checked without this source repository, using curl and a SHA-256 tool. The other two commands document checks I run in this site’s private checkout.
-
curl -s https://hh0hh.com/rules.json | sha256sumHashes the public rule catalogue for comparison with the demo’s snapshot identifier. This verifies which rule bytes were served, not whether the rules or findings are correct. -
npm run releaseIn this site’s private source checkout: blocks release on a stale rule digest, out-of-date share card or unresolved output placeholder. It is not a command a visitor can run without that source. -
node scripts/check-placeholders.mjs --releaseAlso requires the private checkout. This runs the placeholder check by itself; a recorded negative probe restored a violation to dist/ and confirmed exit 1.
The same move, three times — one instrument each:
They are not all enforced to the same standard, so each says which:
- ENFORCED In the configured project or host, the check runs on a commit, CI job or tool call. Its stated conditions and coverage still apply.
- RUNNABLE A real criterion that really fails — but someone has to type the command.
- STATED Followed, but unchecked. If I stopped, nothing would go red.
PROJECT EVIDENCE & ACCESS
nonconstant’s verification code and design notebook, and the historical DevLoop source, are public. Verdict and Terrarium source remain private; their examples below are author accounts with the public demonstrations and recorded evidence available on this site.
Execution claims need inspectable records
A claim about execution needs an inspectable record of execution. Records can still be incomplete or misread; an account of a check is not a substitute for running it.
Verdict
The rule engine is synchronous by contract. Adding `async` fails the call sites at compile time, and the pre-commit hook stops the commit. ENFORCED
DevLoop
When its host hooks are registered and working, a regex sets a mode token and selected quota-spending commands require it. Missing or broken hooks can fail open. ENFORCED
Terrarium
A separate numpy implementation supplies checkpoints and structural comparisons. It is distinct from the repeated-engine fingerprint test and does not require identical long-run populations. RUNNABLE
The essay this rule comes from had to solve its own version of the problem: an argument against trusting machine self-reports, assembled almost entirely from machine self-reports. Every line of evidence in it carries a tier — machine fact, or account.
The guard is guarded
A guard I trust and a guard that works are different things. Deliberate failure cases help test the difference, within each check’s stated scope.
Verdict
`build` sits inside the hook, because a report once said it passed there — in a commit whose hook had no build step. ENFORCED
DevLoop
A gate counts the checks that ran and passed. Zero verified checks is a broken gate — a third outcome, not a pass. ENFORCED
Terrarium
Ten test files declare expected assertion counts and compare them with the executed total, so a test that stops early cannot pass merely by printing no failure. RUNNABLE
One recorded probe failed to apply its deliberate break, so nothing was broken and the green result proved nothing about rejection. That is why I check that the intended failure actually occurs. It is still not enough: a probe that fails to start also exits non-zero. One recorded round of four red checks turned out to be four startup failures.
“I didn’t check” may never render as “it’s fine”
Not-checked and checked-clean are the same exit code until you force them apart. So a verdict that does not name what it skipped is itself a false green.
DevLoop
A constitution registering no unjudgeable clause refuses to load. Not a warning — a config error, with the reason in the message. ENFORCED
Terrarium
Unjudgeable clauses are a table, and every verdict carries an “N not covered” tail. Inventing a heuristic for them is banned. RUNNABLE
This site
The evaluation-conditions row renders only when model, temperature, prompt version and run count are present. The score itself is separate; this checks metadata presence, not truth. ENFORCED
Verdict carries the written counterpart: a list of what its automation cannot verify — regulatory semantics, prompt adherence after a model upgrade, credentials committed by hand.
Whoever does the work does not supply what judges it
Reading the gate and planning to route around it are indistinguishable byte for byte, so where it works the answer is to make the route-around ineffective rather than to detect intent. Where it does not work — anything that reaches the filesystem by a path the guard never sees — the honest fallback is detection, and saying so is cheaper than pretending the guard is a sealed box.
DevLoop
The gate runs from a fingerprinted copy outside the worktree. Caches rebuild, and acceptance uses a clean checkout to identify the tested revision—not to guarantee its correctness. ENFORCED
Verdict
Reviewer profiles declare a read-only tool set. Actual enforcement depends on the host configuration; the declaration is not a security sandbox or proof of execution isolation. STATED
Terrarium
Tests and answer keys are treated as protected criteria. Changing a baseline needs review: a rewritten answer key can produce a normal-looking score. This is not filesystem isolation. RUNNABLE
Every conclusion ships with its range
A result without its scope is not a result. What it covers, what it explicitly does not, and what evidence would overturn it — and a qualifier one click away from its claim is not a qualifier.
Terrarium
Four fields per ruling: what it covers, what it does not, the basis, and what would overturn it. The last two are the ones that do the work. STATED
Verdict
Two recorded runs of one document set differed. I distinguish rule evaluation on fixed validated inputs from model extraction; end-to-end reproducibility was not established by those runs. STATED
This site
A ledger row may state a mechanism and never a score, because a score needs its badge, its scope note and its limitations beside it, and one line has room for none of them. STATED
I write my own ticket first
Corrections ship even when they make the headline worse, and an unknown cause is written down as unknown rather than given one.
DevLoop
A one-line deduplication bug moved output tokens 30×. The correction turned my own headline result from a win into a loss. It shipped. STATED
This site
This page once printed a rule ID, a hash and a PASS in its source. All three were invented. It is now a digest you can recompute. ENFORCED
Terrarium
A 20× speedup is on file as a rejection: faster, but not the same world. Wrong history is struck through, never deleted. STATED
Verdict keeps a section titled “what was explicitly not built”, listing the features whose interface exists and whose backend does not — including the one a demo could most easily gloss over.