Is Prompt Engineering Over, and Is Loop Engineering What Actually Ships?

Two years ago the job title was prompt engineer and the deliverable was a paragraph. Open the documentation published now by the companies that actually sell these models and you will find pages about tool schemas, stopping conditions, guardrails, resumable approval flows, trajectory evaluation, and test files that fail a build, and comparatively little about how to phrase a request. That shift is what happens when a model stops answering once and starts acting repeatedly before a person looks at anything. I ship software with these tools, and I was trained as an accountant, so I read the change the way I would read a change in a control environment: the risk moved, and the controls have to move with it.

What did prompt engineering actually buy us?

It bought real things, and the record is documented well enough to check. A 2024 systematic survey of the field assembled 33 vocabulary terms, a taxonomy of 58 text prompting techniques, and 40 more for other modalities, which is not the profile of a fake discipline.¹⁶ Individual techniques produced measured gains that were not small. Sampling several reasoning paths and keeping the most consistent answer, rather than taking the first greedy decode, improved accuracy by 17.9 points on GSM8K, 11.0 on SVAMP, and 12.2 on AQuA.¹⁴ Those are the sort of numbers that justify a headcount.

The shape of the interaction those techniques assumed is the part worth noticing. You ask, the model answers, you read the answer and decide what to do with it. One exchange, one judgment call, and the judgment call sits entirely with the person at the keyboard. Every safeguard in that arrangement is a human being reading output, which works the way a manual reconciliation works: it is slow, it does not scale, and it fails quietly the first time the reviewer is tired.

The same survey that catalogued all those techniques also said the field suffers from conflicting terminology and a fragmented understanding of what makes a prompt effective.¹⁶ That is an unusually honest thing for a survey to say about its own subject. It describes a craft with results and without theory, which is exactly the kind of craft that gets absorbed the moment the underlying tool improves.

Why did the model vendors stop writing about sentences?

Because what they now have to explain is what happens after the request rather than the request itself. Anthropic's engineering guidance draws the line plainly: workflows are systems where models and tools are orchestrated through predefined code paths, and agents are systems where the model dynamically directs its own processes and tool usage.¹ The patterns it documents are structural rather than linguistic: prompt chaining with programmatic gates between steps, routing, parallelization, an orchestrator delegating to workers, and an evaluator paired with an optimizer so one model grades what another produced.¹

OpenAI's agent documentation describes agents as applications that plan, call tools, collaborate across specialists, and keep enough state to complete multi-step work, with guardrails that block or pause before risky work continues, and resumable approval flows for the moments a human has to say yes.³ Google's function calling documentation is blunter about the mechanics and better for it: the model does not execute the function itself, your application extracts the name and arguments and runs it, and you should validate function calls before executing.⁵ Anthropic's tool use documentation describes the same round trip. The model returns a structured request, your code executes the operation, and you send the result back in the next turn.²

Read those three together and the same structural fact repeats: the model emits an intention, and your code decides whether to honour it. Everything that matters about safety, cost, and correctness lives in that gap, and nothing in that gap is a sentence. It is a schema, a validation step, a permission, a retry rule, and a log.

What is actually inside a loop, in concrete terms?

Strip out the vocabulary and a loop is four moves repeated: the system acts, observes what happened, checks the result against a defined standard, and then stops, corrects, or tries again. The research on this is older than the current enthusiasm. The Reflexion framework had agents reflect verbally on task feedback and keep that reflection in an episodic memory buffer to inform later attempts, reporting 91 percent pass at one on HumanEval against a prior state of the art of 80 percent, without touching model weights.¹⁵

Google's agent development documentation is the clearest published statement of why this needs its own test discipline. Because of the probabilistic nature of models, deterministic pass or fail assertions are often unsuitable, so you evaluate two separate things: the trajectory, meaning the steps and tools the agent chose, and the quality of the final response.⁶ It ships two file formats for this, individual test files for active development and evalsets for multi-turn integration testing, and it notes that conformance testing fails if things do not match, which makes it useful in a continuous integration pipeline.⁶ A loop is not real until a bad result can block a deploy.

So when someone tells me they have built an agent, these are the things I want to see, and none of them are impressive to look at:

• An evaluation set. A fixed collection of inputs with known good outputs that runs on every change, not a demo someone drove by hand last week.
• A regression check. Evidence that the last three changes did not quietly break something the system used to get right.
• A stopping condition. A maximum number of iterations or a defined failure state, because a loop with no limit amplifies both errors and cost.¹
• A rollback. A tested path back to the previous behaviour that does not require the person who built it to be awake.
• A human review step. A named person who approves anything consequential before it reaches a customer, a ledger, or a public page.

Why is the verifier the hardest part to build?

Because the obvious way to build it does not hold up. The obvious way is to have a model grade the output, and there is real support for that. The paper that introduced MT-Bench and Chatbot Arena found that strong model judges matched controlled and crowdsourced human preferences at over 80 percent agreement, which is the same level of agreement humans reach with each other.¹¹

That is the headline, and the rest of the same abstract names the failure modes itself: position bias, verbosity bias, self-enhancement bias, and limited reasoning ability.¹¹ A separate paper made the position problem concrete in a way that is hard to forget. Simply changing the order in which two candidate answers appeared was enough to skew the result, letting a 13 billion parameter open model beat a far stronger commercial model on 66 out of 80 tested queries with that commercial model acting as the judge.¹² That is not measuring quality; it is measuring seating position.

Then there is what happens when you ask a model to check itself. Researchers examining intrinsic self-correction, meaning correction based solely on the model's own capabilities with no external feedback, found that models struggle to self-correct reasoning without external feedback and that performance sometimes degrades after the attempt.¹³ I would put that in plainer terms. A loop whose only verifier is the same model that produced the work is the person who wrote the cheque signing the approval line underneath it. In an audit that fails on sight, and it should fail here too.

None of that means model-assisted evaluation is worthless. It means it is one input, it has known and measurable biases, and it needs the same treatment any biased measurement gets: randomise the order, hold out a sample for human scoring, and track agreement over time so you find out when the judge drifts.

Does retrieval fix the facts, or just move the problem?

Retrieval is the other half of what people mean by the loop, and it is worth being precise about what it does. The original 2020 paper combined a pretrained sequence to sequence model with a dense vector index over Wikipedia, set a new state of the art on three open domain question answering tasks, and produced language the authors described as more specific, diverse, and factual than a parametric-only baseline.⁹ The same abstract also named what remained unsolved: providing provenance for decisions, and updating a model's world knowledge.⁹ Six years on, those are still the two things that break in production.

The most useful account of how they break comes from an experience report across three separate domains, which named seven distinct failure points.¹⁰ They are worth listing because each one fails differently and each one needs a different check. Content that is simply missing from the corpus. Relevant documents that exist but rank too low to be retrieved. Documents retrieved but dropped during consolidation before they reach the context window. Answers present in context but not extracted because of noise or contradiction. Correct content in the wrong format. Answers pitched at the wrong level of specificity. Answers that are right but incomplete.¹⁰

That paper also reached a conclusion I think about more than any other line in this literature: validation of a retrieval system is only feasible during operation.¹⁰ You cannot certify it at design time the way you certify a deterministic function. An accountant knows exactly what that implies. A control you can only test while it is running is a control that requires continuous monitoring, and the cost of that monitoring belongs in the business case from day one, not as a surprise in month four.

What is the strongest version of the argument against me?

There are two, and both have merit. The first is that prompting did not get replaced, it got absorbed by capability. The best evidence for this comes from a vendor arguing against its own former advice. OpenAI's guidance for reasoning models tells developers to avoid chain-of-thought prompts, on the grounds that because these models reason internally, telling them to think step by step or explain their reasoning is unnecessary. It advises keeping prompts simple and direct, and trying zero shot before reaching for examples.⁴ A technique that earned 17.9 accuracy points a few years ago is now something the vendor tells you to stop doing. That is what capability absorption looks like, and anyone claiming prompting is a permanent skill should sit with it.

The second is that loop engineering can be a consultant's rebrand, and often is. The most useful check on that is again from a vendor, which has every commercial reason to sell complexity and instead advises against it. Anthropic's guidance says to add complexity only when it demonstrably improves outcomes, that for many applications optimizing single model calls with retrieval and good examples is usually enough, and that agents trade latency and cost for task performance, which is only worth it when that trade actually matters.¹ If someone quotes you a multi-agent architecture for a job a single well-scoped call handles, they have sold you overhead and called it engineering.

So the honest position is narrower than the slogan. Prompting still decides outcomes at every step inside a loop, because every step is a prompt. What changed is that it stopped being the whole job and became one component with a known failure profile, sitting inside a system that has to catch its failures.

How much of any of this reaches production?

Very little, and the gap is worth putting numbers on. Stanford's 2026 AI Index reports that 88 percent of surveyed organizations have adopted AI in at least one capacity, with roughly 70 percent using generative AI in at least one business function. It also reports that AI agent deployment remains in the single digits across nearly all business functions.⁷

Near universal adoption of models sits alongside near zero deployment of systems allowed to act on their own. That is a controls gap rather than a capability gap, because the same report documents agent task success on a desktop benchmark rising from 12 percent to roughly 66 percent.⁷ The models can act. Almost nobody has built an environment they would be willing to sign their name under, and the people declining to sign are correct to hesitate.

That is an accountability problem, and there is a published framework for it. The NIST AI Risk Management Framework, published in January 2023 and explicitly voluntary, organises the problem into four functions: govern, map, measure, and manage.⁸ Read it as an operator rather than a compliance officer and the sequencing is the lesson. Govern comes first. You decide who is accountable before you decide what to measure, because you cannot measure your way to an owner.

What would I require before signing off on one?

The same things I would require before signing anything else. Define what done means specifically enough that a machine can test it, because a standard that only exists in someone's head cannot be tested at all. Keep an evaluation set that runs on every change and blocks a deploy when it regresses, in the way conformance testing is meant to work.⁶ Put a hard stopping condition on every loop, and treat an unbounded retry as a cost exposure rather than a robustness feature.¹ Validate every tool call before it executes, because the model is proposing an action, not performing one.⁵ Route anything consequential through an approval step with a named human on the other side.³ Assume your retrieval layer will fail in at least seven distinguishable ways and monitor for them while it runs, since you cannot certify it beforehand.¹⁰ And never let a model be the sole judge of its own work.¹³

None of that is a reason to be timid with these tools. I use them ambitiously and I would recommend that anyone building right now does the same, because the capability is genuinely there and the productivity is real. But ambition without a verifier is just speed toward an unreviewed result, and speed toward an unreviewed result is how a firm ends up explaining something in writing to a regulator or a customer.

So if someone offers to sell you the prompt, ask what happens after the model responds. If the answer is that you read it and decide, you bought a prompt, and there is nothing wrong with that as long as you know it. If the answer involves a defined check, a retry rule, a stopping condition, and a person who owns the outcome, you are looking at a system. The second one costs more to build and it is the only one that survives contact with work that matters.