Prompts are interfaces, not incantations

There is a tell in how a team talks about its prompts. If the language is “we found some wording that works” — if the prompt is a string someone tuned in a playground, pasted into the codebase, and now nobody dares touch — then the team is treating prompting as incantation. The words are magic. Nobody knows which words are load-bearing. Changing them is folklore-driven risk.

The corrective is to notice what a prompt actually is. It is the interface between your deterministic system and a probabilistic component: the specification of behaviour you hand to the model on every call. It carries the same weight as an API contract or a database schema, with one aggravating difference — the component on the other side does not throw a type error when you violate it. It just behaves slightly differently, and you find out downstream.

Interfaces get engineering discipline. Incantations get superstition. This post is about what the discipline looks like in practice.

Prompts live in version control

The first failure mode is architectural, not linguistic: the prompt lives somewhere other than the repository. A database row edited through an admin panel. A dashboard field in a vendor console. A constant someone updates in production “just to try something”.

The consequences are predictable. You cannot diff what changed between Tuesday’s behaviour and Wednesday’s. You cannot correlate a quality regression with the edit that caused it, because the edit left no trace. You cannot roll back, because there is no previous version to roll back to. Every property you rely on for code — history, blame, revert, branch — you have silently given up for the single artefact with the most behavioural leverage in the system.

The fix is unexciting and complete: prompts are files in the repository. They are deployed like code, through the same pipeline, with the same environments. A prompt change to production goes through a pull request, and the deployed system reports which prompt version it is running, the same way it reports a git SHA. Prompt management platforms are fine as a distribution layer; they are not a substitute for the source of truth living under version control.

Separate instruction, context, and data

A prompt that reaches the model is usually three things interleaved, and most prompt bugs come from failing to keep them distinct:

Instruction — the behavioural specification. Who the model is acting as, what it must do, what it must never do, the output format. Written by engineers, changed deliberately, versioned.
Context — information the system supplies for this call: retrieved documents, user profile, prior conversation, tool results. Assembled at runtime, variable, and only partially trusted.
Data — the payload being operated on: the email to classify, the document to summarise, the user’s message.

Keeping these in one undifferentiated string causes two distinct classes of failure. The first is maintenance: nobody can safely edit instructions that are braided through interpolated runtime content. The second is security: if the model cannot tell where instructions end and data begins, neither can it tell that a sentence inside a retrieved document saying “ignore your previous instructions” is data rather than instruction. Prompt injection is, at root, an interface-separation failure.

The practical pattern is a template with explicit, delimited sections — instructions in one block, each piece of runtime context in its own labelled and fenced region, the payload clearly marked as untrusted material to be operated on rather than obeyed. The template is the versioned artefact. The rendered prompt is an ephemeral product of template plus inputs, and your traces should record both.

[instructions]        versioned, reviewed, deliberate
[context: retrieval]  runtime, provenance-tagged, semi-trusted
[context: history]    runtime, summarised by the harness
[data: user input]    untrusted, delimited, never treated as instruction

A prompt change is a behaviour change

Here is the argument that justifies everything else in this post. When you change a prompt, you have changed the behaviour of the system for every input it will ever see. That is exactly what a code change does — except a code change is constrained by types, tests, and locality, while a prompt change has none of those constraints natively. A one-word edit can shift refusal rates, output format compliance, tone, and tool-selection behaviour simultaneously, across use cases the editor was not thinking about.

This is why the edit-in-a-playground workflow is so dangerous. The playground shows you one input. The edit ships to all of them. The author checked that the change fixed the case in front of them; nobody checked what it did to the cases that were previously working. Prompt regressions are almost never noticed at edit time — they surface as a slow drift in production metrics, weeks later, unattributed.

The consequence: no prompt change merges without eval coverage, full stop. Concretely, that means:

A regression suite of representative inputs with graded expectations — exact-match assertions where outputs are structured, model-graded or rubric-based scoring where they are open-ended. The suite runs on every prompt PR, the same trigger as unit tests.
Case-level diffing, not just aggregate scores. A change that moves the aggregate from 91 to 92 while flipping eight previously-passing cases to failure is not an improvement; it is a trade you need to see before you accept it.
Adversarial and edge cases in the suite: injection attempts, malformed inputs, out-of-scope requests. Prompt edits routinely loosen refusal behaviour by accident.
Every incident becomes a case. When production surfaces a failure, the input joins the suite before the fix merges — the same discipline as regression tests for bugs.

If a prompt matters enough to ship, it matters enough to measure. A team that cannot afford eval coverage for a prompt cannot afford the prompt.

Reviewing a prompt like code

Prompt review fails when reviewers treat it as copy-editing. The useful review questions are behavioural:

What behaviour is this change intended to produce, and which eval cases demonstrate it?
What existing behaviour could it plausibly degrade, and did the regression suite cover that surface?
Is any new instruction in conflict with an existing one? Contradictory instructions do not fail loudly; the model silently arbitrates, and the arbitration varies by input.
Is anything in the diff superstition? “You are the world’s leading expert”, triple-emphasised ALL-CAPS warnings, ritual phrases inherited from a blog post in 2023 — every clause should earn its place with a measurable effect, because every clause consumes attention and tokens on every call. If removing a sentence changes nothing in the evals, remove it.
Does the change belong in the prompt at all? Output structure is often better enforced by schema-constrained decoding; authorisation belongs in the harness; retries belong in the runtime. The prompt should carry what only the prompt can carry.

Small diffs, one behavioural intent per change, evals attached. The same review values as code, because it is subject to the same failure physics as code.

The test of maturity

A useful audit question for any team running LLM features: if quality dropped last Thursday, could you produce the exact prompt that was live, diff it against the previous week, and identify which change shipped without eval coverage? If yes, your prompts are interfaces. If no, they are incantations — and the system’s most leveraged artefact is being managed with less rigour than its logging config.

The encouraging part is that none of this requires new machinery. Version control, templating, review, regression suites — the tools already exist, and the habits already exist for code. The work is deciding that prompts are inside the engineering perimeter rather than outside it. Teams that make that decision stop being afraid of their own prompts, and unafraid teams iterate faster than superstitious ones.