What is Context Engineering for AI agents?

What is Context Engineering?

Context Engineering is the discipline of assembling everything an AI model needs to complete a task, the instructions, retrieved data, tools, examples, memory, and business rules, in the right format and at the right moment. It moves the craft from writing a better sentence to designing a better input. The premise behind it is blunt: frontier models are no longer the bottleneck. Two teams using the same model get very different results, and the gap comes from the context each agent is given, not from the wording of a single prompt.

Andrej Karpathy defined it as "the delicate art and science of filling the context window with just the right information for the next step."

For an agent that works with company data, the "right information" is not only documents pulled in at query time. It is the meaning of the data: what an active customer is, which date counts as revenue, that a cost field is stored in millionths, that two records are the same company. When that meaning is engineered into the system, the agent reads a rule. When it is missing, the agent invents one, and it does so with total confidence.

The term went mainstream in June 2025. On June 19, Shopify CEO Tobi Lutke posted that he preferred "context engineering" over "prompt engineering," calling it the art of providing all the context for a task to be plausibly solvable by the model. Karpathy amplified it days later, and Walden Yan at Cognition, the team behind the Devin coding agent, had been writing about the same practice earlier that year. June 2025 gave an old data problem a name that stuck.

Context Engineering vs prompt engineering vs RAG

Context Engineering is the superset. Prompt engineering is a subset of it, and RAG is one component of it. They are not competing choices; they are nested layers of the same problem.

ApproachWhat it optimizesScopeThe question it asksPrompt engineeringThe wording of one instructionA single static stringHow do I phrase this so the model responds well?RAGWhich data enters the windowOne retrieval stepWhat should I put in front of the model right now?Context EngineeringThe whole information environmentInstructions, retrieval, memory, tools, token budget, business semanticsWhat does the model need to answer correctly, and where does each piece come from?

The distinction is not academic. Rewording a prompt cannot fix a number that is wrong because the agent never knew a currency was stored in millionths. Retrieving the deals table is retrieval, but knowing that a merged, absorbed record must not be counted twice is context. An agent needs all three, and the two most people skip are the business semantics and the governance around them.

What are the building blocks of Context Engineering?

It spans everything that enters the context window, plus the systems that decide what belongs there. Six components carry most of the weight:

• Instructions and role: what the agent is for and how it should behave.
• Retrieved data (RAG): the documents or records relevant to the current task.
• Business semantics: the definitions and rules that give data meaning, the part that is most often skipped and breaks results most quietly.
• Memory and state: what persists across turns so the agent does not start blind each time.
• Tools and orchestration: which tools the agent can call, and in what order.
• Token budget: what fits in the window, and what gets compressed or dropped.

For a data agent, the semantics block is the one that decides whether the answer is trustworthy, and it is the one that cannot live in a prompt, because it has to apply to every question, every agent, and every report at once.

Why do AI agents need Context Engineering?

Because an agent, unlike an analyst, does not ask when it is unsure. It answers with confidence even when it is wrong, and a well written wrong answer is far harder to catch than an obvious one. A person who opens a table with a strange column name asks a colleague. An agent deduces. Context Engineering removes the guess by putting the rule next to the data.

The cost of skipping it is now visible at the portfolio level. Gartner projects that over 40% of agentic AI projects will be canceled by the end of 2027, citing rising costs, unclear business value, and inadequate controls. Rising cost and unclear value are precisely what an agent produces when it has to reconstruct meaning at query time instead of reading it. The failure is rarely the model; it is the work being pushed into the model that was never model work.

What goes wrong without Context Engineering?

The agent invents definitions, joins data by reading it, and produces numbers no one can trust. Four failure modes recur, and none of them appears in the demo; they appear in month three of production:

• It invents the definition. Asked "how many active customers," with no rule it counts live subscriptions on Monday and won deals on Wednesday. Two defensible numbers, both different, and the reader goes back to the spreadsheet.
• It joins in the wrong place. Combining tables is database work: keys, indexes, computation. Doing it by reading records inside the context window is expensive and fragile, and it is the main driver of token cost.
• Scope leaks. Without per-agent permissions at the data layer, access control is a sentence in a prompt, which is a request, not a boundary.
• Cost grows with volume, not with the question. The dataset doubles, the token bill doubles, the answer is no better. This is where projects die on budget rather than on engineering.

The concrete cases are unglamorous and decisive. Each one is context about the data, not the data itself:

The caseWhat the agent does wrongThe rule that fixes itAd cost stored in millionthsReports a cost per acquisition a million times too high, explained convincinglyOne unit is 1,000,000; divide before any cost metricCorporate vs personal emailBlends the two and calls a healthy channel brokenCorporate converts at 15 to 20%, personal near 1%; split before measuringHistorical win after churnReports "never a customer" about someone paying todayA churned customer still carries the historical win; count itCreated, closed, first-charged datesAnswers "how much did we sell in June" three different waysPick one revenue date and define it once

None of that is data. All of it is context about the data.

Where does Context Engineering live: the prompt or the data layer?

For data-heavy agents, most of it belongs in a governed data layer, not the prompt. A rule written into a prompt applies to one agent and one conversation, and it drifts the moment someone edits the wording. A rule written into the data applies once, everywhere, to every agent and every report. A layered model makes this concrete:

• Raw keeps what the source sent, unchanged, so any number can be audited back to its origin.
• Trusted holds the cleaning no one sees and everyone depends on: deduplication, correct types, correct time zones, merged-record handling.
• Service is the data modeled in business language, one table per question the company actually asks. This is where the agent reads.

On top of that layer, meaning is annotated per column and served with the data. That annotation, the note that says this cost is in millionths or this date is first-charge, is Context Engineering in its most literal form: the analyst's knowledge, written down, versioned, and delivered to the agent instead of kept in someone's head.

How do you implement Context Engineering for agents?

Start from the questions the business asks, not from the tools. A sequence that works:

1. Write the ten questions the business actually asks every week. That, and nothing more, is the scope to model first.
2. For each, list the sources and the key that joins them. If no shared key exists, that is the first problem, and it predates any AI.
3. Land everything raw and unchanged, so it can be audited and reprocessed.
4. Write the tedious rules once, in versioned code: deduplication, time zones, types, merged records, currency units. Not in a prompt.
5. Model the serving tables in business language, one per question.
6. Annotate context on every column a human would have to ask about.
7. Only then connect the agent over MCP, with per-agent permissions and minimum scope.

MCP is the last step, not the first. Starting there, with no semantic layer underneath, is why so many agent projects stall.

Common mistakes in Context Engineering

The recurring anti-patterns are easy to name and expensive to live with:

• Putting business rules in the prompt. They apply to one agent, drift on every edit, and never reach the next agent that needs them.
• Treating RAG as the whole job. Retrieval without governed semantics feeds the agent clean-looking data with no rules attached.
• Over-scoping the agent. More tables is not more context; it is more noise, more tokens, and more confident mistakes. Tight scope improves accuracy and lowers cost at the same time.
• Skipping column-level annotation. If a human would have to ask what a column means, the agent will not ask; it will assume.
• Connecting MCP first. A connection protocol with no layer beneath it just makes the guessing faster.

How does Nekt do Context Engineering?

Nekt is a data platform built to give AI agents governed context, which is Context Engineering delivered as a product rather than a project. It covers the three parts an agent needs, in one place:

• Connect. Nekt brings your sources together through ready-made connectors, so the data lands without hand-built integrations.
• Govern. It organizes data into Raw, Trusted, and Service layers, with SQL and Python transformations, and it lets you annotate business meaning per column: how the company measures MRR, what counts as an active customer, which date is revenue. Permissions are set per table and per layer, per agent, so scope is a control, not a sentence in a prompt.
• Serve. Agents reach the result over an MCP server or a Data API. Over MCP, the agent receives more than the table: it receives the definition alongside it, so the business rule arrives embedded in the answer. For questions about the current state of a record, live data reads the source in real time; for anything aggregated or historical, the agent reads the modeled layer.

The point is that the rule is written once and travels with the data to every agent, instead of being re-explained in every prompt. That is why the same question stops returning different numbers, and why an agent can cross many systems in a single query instead of paging through each one.

What results does Context Engineering produce?

The gain shows up as accuracy, cost, and speed on the same model. In one production benchmark, an agent querying sources directly was compared with the same agent on a governed data layer, with context served alongside the data.

MetricQuerying sources directlyGoverned data layer, context servedAccuracy38%91%Work to answer234 tool calls1 SQL queryTime~15 minutes19 seconds

Same model, same question: 38% accuracy over 234 tool calls, or 91% accuracy in one query. The difference is not the model; it is the context beneath it. (Nekt benchmark.)

You can see this in your own data. See how Nekt works, or create a free account and connect your first source.

Frequently asked questions

Is Context Engineering the same as RAG?

No. RAG is one component of Context Engineering. RAG handles retrieval, pulling relevant records into the context window. Context Engineering also covers memory, tool orchestration, token budget, and, for company data, the business rules that make retrieved data mean something.

Does Context Engineering replace prompt engineering?

No. Prompt engineering is a subset of Context Engineering. Wording still matters, but it cannot fix an answer that is wrong because the agent lacked the definition. That is a context problem, not a phrasing problem.

Is Context Engineering the same as a semantic layer?

A semantic layer is a large part of Context Engineering for data agents, but not all of it. The semantic layer supplies governed definitions and models; Context Engineering also includes memory, tools, and how everything is assembled in the window at runtime.

What tools do Context Engineering for AI agents?

The practical answer is a data platform that can connect sources, govern them with per-column meaning, and serve them to agents with the definitions attached, over MCP or an API. Nekt is built specifically for this: connect, govern, and serve context to agents from one place.

Do you still need Context Engineering if you already use RAG?

Yes. RAG gives the agent the right documents; it does not give the agent the rules for interpreting them. Without governed semantics, an agent can retrieve perfectly and still count the same customer twice.

‍