There's a comforting story going around that anyone with an AI can now do anyone's job. This is a small, true counter-example: a live database migration where the AI did most of the keystrokes and none of the judgment — and where the safest thing the fleet did was stop one step short of the end. I directed the whole thing from a phone during the flight home, overnight — and it was safe not because I was watching, but because the guardrails were built for exactly that.
The ask that looked simple
An atomic-clock lab's entire telemetry pipeline ran through a database on an unreliable mini-PC. When it slept or rebooted — which it did — data capture stopped and the lab's analysis tools went dark. The request was one line: "Move the database to the new box. Don't lose data. Don't disturb the running experiment."
What it actually required was a zero-data-loss migration of a live time-series database — cross-operating-system, feeding a running physics experiment, with a scientist's tools authenticating to it by tokens that had to keep working afterward. "Move the database" was really "move a system that can't be turned off, prove it's byte-for-byte identical before you commit, and never break the thing measuring the clock."
The AI can type the migration faster than any of us. It cannot tell you that the rollback design it just proposed will quietly take the live system down.
Five places the tool would have failed alone
Each of these is a decision, not a command. At each one, an AI left to its own devices would have produced something that worked in the demo and broke in the field. Knowing that — and steering around it — is what expertise is.
-
The AI's own plan had a live-system-killing bug.
An early revision proposed two-way replication for a clean rollback. A second, independent AI reviewer proved it forms a replication loop that never drains — it would have thrashed the live system. It was removed before a byte moved. That's the whole point: one AI builds, a different one attacks, a human arbitrates.
-
"Zero downtime" would have been a lie.
The live writer can't buffer across its own restart. Rather than promise impossible zero-interruption, I made the switch an announced, bounded, seconds-scale gap — strictly better than the silent drops happening today — and said so in plain language. Naming the limitation honestly is the deliverable.
-
The last step had to stay human.
Even told to "build all you can overnight," the fleet built, rehearsed, and validated everything up to the one irreversible action — repointing the running clock's data writer — and stopped. That step needs a person on-site and a battery backup in place. Autonomy is only useful if it knows where to end. The parallel run is now live in production — the database is fully copied onto the appliance and mirroring in real time; only that one flip remains. Further out, the roadmap's control phase turns the same appliance into a control host — eventually driving multiple clock chips from one place, once the lab's hardcoded per-device IPs are made configurable, with the clock hardware controlled last and behind the guardrails.
-
"It works" had to be proven, not promised.
Nothing that had to be exactly right was left to the model's word. A byte-level parity check proves the copy is identical; an integer-exact time-window filter — a floating-point version silently passed out-of-range data, and the review caught it — guards the boundaries. Money, measurements, and migrations run in deterministic code with tests, not vibes.
-
A production credential the AI couldn't take on its own.
Fetching a production admin token was automatically declined until I explicitly authorized it — the safety guardrail firing exactly as designed, on a real secret, mid-migration. Credentials never touch a prompt; sensitive actions become approvals.
What shipped
The model: expertise directs the AI — and keeps the meaning
The way I work now: I'm the product owner; the AI is the engineer and the QA. It executes and it tests. But I decide what's worth doing, I catch the confident-but-wrong answer, and I keep every irreversible, high-stakes call — money, security, production, relationships — under human review. Reversible, high-volume work runs on autopilot. That division is the entire skill.
Guardrail 0 · the prime one
A knowledgeable human owns the loop.
The other five are mechanisms; this is what makes them real. Someone who knows what “right” looks like decides what counts as irreversible, what must be deterministic, whether a proposed change is safe to approve, and whether the logs are telling a bad story. Remove this one and the rest are theater — AI becomes a confident way to break things faster. It is the difference between the top and the bottom of the completion table.
The five guardrails that make it leverage, not liability
- Humans hold the irreversible calls. The system handles routing and volume; a person owns anything hard to undo — money, production deploys, relationships, anything customer-facing.
- Determinism where it matters. Anything with an objectively correct answer — financial math, measurements, logic — runs in deterministic code. AI handles only language and judgment, never the numbers.
- Credentials never touch a prompt. Secrets live in a vault and resolve at runtime — never pasted into a model, a log, or a chat history.
- AI proposes; humans approve. Destructive or irreversible actions become a pull request or a draft for review — never applied directly. Rollback is always available.
- Everything is logged and cost-capped. Every action is auditable and replayable; per-agent and per-session spend limits mean no runaway loop can burn the budget.
That's why a project like this can't come from "pushing on an AI." The leverage is real — but it's leverage on expertise, not a substitute for it. The person still has to know what right looks like, and know when the machine is wrong.
The one assumption everything rests on — and the way out
Every guardrail rests on Guardrail 0: a knowledgeable human in the loop. Take that away and the rest collapse. That is the "True Believer" — following AI confidently with little knowledge to check it, running the mechanisms without the judgment that powers them.
What it looks like:
- Approves the destructive change because it looked fine.
- Leaves the service on a public port because the AI never mentioned it.
- Pastes a secret into a prompt to move faster.
- Accepts a confident-but-wrong answer and ships it — then declares it done.
The danger isn't slowness. It's undetected wrongness, shipped fast, at the highest blast radius — because no one is reviewing.
The way out. Here's the hopeful part: this is a position, not a person — and it's the most fixable one on the list. You don't have to become the world's expert; you need just enough judgment to QA the tool and to know when to slow down.
- Treat AI as a proposer, not an oracle — make it explain why, show the alternatives, and say what could go wrong.
- Verify with something deterministic — does it build, does the test pass, did the number actually move? — instead of trusting a confident tone.
- Put a pause in front of anything irreversible: "what breaks if this is wrong?" before you run it.
- Learn in public with the tool: ask it to teach you the thing it's doing, not just do it. Every task becomes a lesson, and the knowledge compounds.
- Borrow a reviewer for the high-stakes calls until your own judgment catches up.
The trajectory is the point. A True Believer who adds verification and curiosity becomes a competent guide surprisingly fast — AI is the fastest apprenticeship ever built, if you let it teach you instead of just doing for you. The gap between “feels done” and “is done” isn't a life sentence; it's a skill you build, with the very tool that exposed the gap.
The unglamorous foundation is the enabling one
I'm not the physicist making the clock tick — but nobody designs that chip until this server is live, secure, and unbreakable. That's the work I do: stand up the systems that let the hard, high-stakes work happen, and run them so a team never has to think about them. It's the same muscle whether it's an EDA license server, a healthcare platform, or the next AI product that has to be trustworthy on day one.
Need someone who can build the system and run the operation it lives in?
The Senkungu Group →