What is context engineering?

It is the discipline of deciding exactly what information a model is given for each request: which documents, how much history, which instructions, in what order. It matters because a model has no memory between requests and knows nothing about your business, so everything it uses to answer has to be assembled and handed to it every time. The quality of that assembly usually affects the answer more than the choice of model does.

Does giving AI more context produce better answers?

Only up to a point, and past that point it makes things worse. Handing a model fifty pages to answer a question that needed two paragraphs increases the chance it attends to the wrong part, and information in the middle of a long input is attended to less reliably than information at either end. It also costs more and takes longer. The goal is relevance rather than volume: the smallest set of material that fully answers the question.

Why does the same AI question give different answers?

Usually because what was handed to the model differed, even when the question looked identical. A different document was retrieved, a longer history was included, a template inserted something extra. This is why systems that behave well in testing can degrade in production: the test cases came with clean, short context and real cases arrive with more of it, assembled differently.

How does context affect the cost of running AI?

Directly, because providers charge by how much text goes in and comes out. A step handed the whole document each time instead of the relevant section can cost many times more for the same result, and in a multi-step workflow where each step passes everything forward, the accumulated input grows at every stage. Cost per item rising without volume rising is usually a context problem instead of a pricing one.

Context Engineering: Why What You Feed AI Decides the Answer

The same model. The same question. Two different answers, and nothing about the question changed.

What changed was what the model was handed. One version received the relevant clause. The other received the whole forty-page agreement and had to find it, and did not quite.

That difference has a name, context engineering, and it turns up in three disguises. As a cost lever, because handing a step two paragraphs instead of fifty pages changes the bill by an order of magnitude. As an architecture principle, because each step should get what it needs and nothing more. And as an unexplained term in lists of things that are apparently hard.

All three are the same decision. The piece on orchestration puts it in one line, that each step needs exactly what it needs, and then moves on. This is what deciding that actually involves, and it is probably the highest-return part of building AI that nobody outside the work talks about.

Why anything has to be handed over at all

A model has no memory between requests and no knowledge of your business. Every request arrives with no history. Whatever it uses to answer has to be assembled and handed to it, every single time.

That assembly is the context, and it typically contains four things: the instruction describing the job, any examples of what good output looks like, the material relevant to this particular case, and whatever the conversation or workflow has established so far.

Everything a system knows in the moment of answering is in that bundle. Nothing else is available to it. Which makes the composition of the bundle the single most consequential design decision in most builds, and it is made by whoever wrote the code, usually without anyone reviewing it.

More is worse, past a surprisingly early point

The intuitive move is to include everything and let the model sort it out. The window is large, the material might be relevant, why risk leaving something out.

Three things go wrong.

Attention thins. A model given a great deal of material distributes its attention across all of it. The relevant clause competes with thirty-nine pages of irrelevant ones. On most models tested, information positioned in the middle of a long input is attended to less reliably than information near either end, so a critical detail can be present and effectively unread. The size of that effect varies by model and has narrowed on recent long-context ones, which makes it a thing to test instead of assume.

Contradictions arrive. Fifty pages of accumulated material frequently contains two things that disagree: an old policy and its replacement, a quoted price and a revised one. A model handed both will use one of them, and which one is not predictable. This is the same failure that makes an unsorted document set produce confident wrong answers, arriving by a different route.

The bill grows. Providers charge by volume of text in and out. A step handed the whole document each time rather than the relevant section costs many times more for the same result, and in a sequence where every step passes everything forward, the input grows at every stage. This is a large part of why an agent can run up a startling bill overnight: the cost is driven by what each step is handed far more than by the headline price of the model.

The cost side is worth doing with figures once, because the multiple surprises people. Take a step answering questions against a supplier agreement. Handing over the whole forty-page document is on the order of 20,000 tokens of input. Handing over the retrieved clause plus its surrounding context is perhaps 800. At a representative $3 per million input tokens that is roughly 6 cents against 0.24 cents per question, a factor of twenty-five. Run that step 4,000 times a month and the difference is about $240 a month, or near $2,900 a year, for output that is also worse in the expensive version. Providers now offer caching for repeated portions of an input at a fraction of the normal rate, which takes some of the edge off, and it does nothing at all for the quality problem.

So the target is not maximum context. It is the smallest set of material that fully answers the question, which is a harder thing to build and a much better thing to have.

The four decisions

Concretely, building the context for a step means deciding four things.

What to include. Which documents, which fields, how much history. This is where retrieval does its work: instead of handing over everything, find the passages that bear on this question and hand over those. The quality of the finding decides the quality of the answer.

How much history. In a sequence, each step could receive everything that happened before it. Almost none of them need it. Passing the full history forward is the most common early design and it degrades quality while inflating cost. What a step actually needs is usually two or three specific facts that earlier steps established, and identifying those is the state problem that makes orchestration hard.

In what order. Position matters, and the direction of the effect varies enough between models that provider guidance disagrees. What holds generally is that an instruction buried in the middle of a long body of material is the least reliable place to put it. Whether the material goes before or after the question is worth testing instead of assuming.

How much instruction. Enough to define the job and the output format, plus an example or two where the format is unusual. Past that, additional instruction competes with the material for attention.

None of these has a universal answer. All of them have a right answer for a particular job, and finding it is done by trying variants against a fixed set of known cases instead of by reasoning about it.

Why testing looks good and production does not

There is a specific failure this explains, and it is common enough to be worth naming.

A system tests well. Thirty cases, high accuracy, everyone is satisfied. It goes live and quality is noticeably worse, and nobody can find a bug.

The usual cause is that the test cases carried clean, short, well-formed context, because they were assembled by hand by someone who knew which document mattered. Real cases arrive with more material, assembled automatically, including things nobody anticipated: an email thread with eleven replies, a document with an unrelated attachment, a customer record carrying nine years of notes.

The model did not get worse. What it was handed did. This is why the test set has to include realistic context rather than idealised context, and why a system can drift as the material around it accumulates while nothing in the system changes at all.

What this means for the person buying

Most of this is engineering, and three parts of it are a buyer’s business.

It explains the cost difference between two quotes. Two builds doing the same job can differ by a large multiple in running cost, and the usual reason is that one assembles context carefully and the other passes everything forward. That difference does not show up in a demonstration and shows up every month afterwards.

It is why “we use the best model” answers the wrong question. The model is roughly the same one everyone can rent. What decides quality is which material gets found, what gets carried between steps, and what each step is asked to attend to. A build that has thought about that will describe it. A build that has not will describe its model.

It is the reason precision in your own instructions matters. The same principle governs the person typing into a tool. A vague question retrieves vague material and produces a vague answer. This connects directly to how meaning becomes a position that gets searched: a precisely worded question lands somewhere precise, and what comes back is what sits nearby.

The question worth asking

For any step in a proposed build: what exactly does this step receive, and why that instead of everything.

A good answer is specific. This step gets the three highest-scoring passages from the contract set, plus the customer’s name and account tier, plus the two decisions made in steps two and four, and nothing else. That is a design, and it will be cheaper and better than the alternative.

A vague answer, that the step gets the relevant information, means nobody has decided, which in practice means it gets everything. That build will cost more per item, produce less reliable output, and degrade as the material around it grows, and none of that will be visible in the demonstration.

The part worth carrying out the door

What a model is handed decides more about the answer than which model it is.

More context is not better context past a surprisingly early point: attention thins, contradictions creep in, and the bill grows at every step. The work is deciding what each step actually needs, carrying forward the two or three facts instead of the whole history, putting the instruction where it will be followed, and testing the variants against real cases rather than reasoning about them.

It is the least visible part of an AI build and one of the most consequential, which is a fair description of most of the engineering that separates a system from a demonstration.

Give It the Two Paragraphs, Not the Fifty Pages