How To Ship Your AI Slop

Andrew Siebert

16 Sep 2026 • 4 min read

"Vibe coding" gets you a working prototype in an afternoon — describe the feature, let the model iterate against your acceptance test, watch it converge. That speed is real, and it's also exactly why the failure mode is so easy to walk into: the same fluency that makes a demo look finished is what makes a reviewer stop looking as hard. A demo that runs clean against the one happy-path input someone tried in the moment tells you nothing about the malformed payload, the network timeout, or the empty array your production traffic will throw at it by the following week.

"Looks Good to Me" Is a Review Failure, Not a Compliment

"The Context Gap" named this failure mode for stale context files, and "The Routing Problem" extended it to a model operating past its capability ceiling: Confident Failure — fluent, structurally correct, and wrong in a way self-consistency checks don't catch, because agreement with itself isn't the same as correctness. The same shape shows up at the review layer for AI-generated code, for a specific reason: code that reads clean gets processed by a human reviewer faster and with less scrutiny than code that reads awkward, regardless of whether it's actually correct. A hand-written pull request with a clumsy variable name draws a comment. Generated code with a well-named variable and a swallowed exception three lines down draws an approval, because the reviewer's attention is calibrated to signals — inconsistent style, an obvious hack — that AI-generated code doesn't emit even when it's wrong.

Critical Rule: Fluency is not a correctness signal. A code review process calibrated to catch code that looks wrong will systematically miss code that looks right and behaves wrong — which is precisely the failure mode AI-assisted code produces most often.

Two Different Problems Wearing the Same Name

"AI dependency" collapses two distinct risks that need different fixes. The first is code the model wrote — static once committed, deterministic in execution, and the risk sits entirely in whether it was reviewed and tested as rigorously as anything else in the codebase. The second is code that calls a model at runtime — a request handler that hits an LLM for summarization or classification as part of its normal execution path — and that risk is different in kind: the call itself is non-deterministic, the response schema is only as reliable as the provider's adherence to it, and every downstream consumer of that response inherits variance the rest of the system was never built to tolerate.

Isolating the Blast Radius of a Runtime Model Call

"Building Custom Guardrails"'s framing applies almost unchanged: a sub-agent scoped to a directory can't cause damage bigger than that directory, no matter what it decides to do. The same logic isolates a runtime AI dependency architecturally. Put every call to a model behind an adapter — a single, narrow interface the rest of the codebase depends on, never the provider's SDK directly — and the blast radius of that call's non-determinism is bounded by the adapter's contract, not by how many places in the codebase happen to import an LLM client. Pin the adapter's output to a schema — structured output or function-calling constrained to a defined JSON shape, validated at the boundary before anything downstream touches it — and a malformed or hallucinated response fails loudly at the adapter, instead of silently propagating a shape nothing downstream was written to expect.

The economics matter as much as the safety here: an adapter boundary is also the one place you can swap which model answers the call — a local model for the routine cases, escalating to a frontier one only when the adapter's own confidence check fails — without touching a single line of the code that consumes the result.

Deterministic Testing Is the Sandbox for Non-Deterministic Code

Unit tests against an adapter's contract, not against the model's actual output, are what make this tractable: assert the shape and the invariants the rest of the system depends on, not the exact string a model happens to generate on a given run. Contract tests at every service boundary the generated code touches catch drift the model itself can't self-report. A golden-file suite — a fixed set of recorded prompts replayed against the current model on every CI run, diffed against last-known-good output — turns "did the model's behavior silently change" from a production incident into a red check on a pull request.

Human code review is advisory — it degrades exactly when reviewer attention is miscalibrated, which is the specific condition fluent AI-generated code creates. A CI gate that fails the build on a broken contract test is structural. Treat AI-assisted code as needing more of the second, not more of the first.

What the Isolation Boundary Doesn't Solve

None of this makes a runtime model call free. Every request that hits the adapter — the routine ones and the escalated ones alike — is a metered cost nobody's actually optimized, and an adapter architected to isolate risk says nothing yet about whether it's routing those calls anywhere close to efficiently.

Finally - A word from the realist

The reality of the situation is that not everything needs to be a polished enterprise level product. Sometimes "AI Slop" is good enough. Hell, plenty of human written slop has been and will continue to be released. Take the time to be realistic about what you're building and where it lands on the scale of "tool only I'll ever use" to "enterprise product".

I'm taking a fully AI-generated ("vibe-coded") side project, refactoring it behind exactly this kind of isolation boundary, and wrapping it in an automated GitHub Actions test harness — live, in a 90-minute session at The Cloud & AI Summit 2026.