Part of
Enterprise context layer
Architecture

What Is Context Engineering, and Why Enterprise Agents Fail Without It

One request built two ways: 100k tokens with everything in the prompt, against 4.1k tokens of engineered context, highlighted.

Context engineering is deciding what an AI model sees at each step of a task. That means choosing which instructions, records, tools and history go into the model's context window, the text it reads on that step. It also means choosing their form, and what stays out. Give an agent too much, the wrong version of a policy or a database schema it cannot make sense of, and it gives wrong answers. A stronger model reading the same context makes the same mistakes. An enterprise context layer makes those decisions once, for every agent that reads your data.

Researchers have now measured this. Chroma tested 18 models in July 2025, including GPT-4.1, Claude 4 and Gemini 2.5, and found that their performance grows increasingly unreliable as the input gets longer. A Microsoft and Salesforce team ran more than 200,000 simulated conversations. The same tasks scored 39% lower on average when the information arrived over several messages instead of one. Models make early assumptions and, in the authors' words, get lost and do not recover. Enterprise agents work in exactly those conditions: long tasks, many systems, and data that changes between steps.

What is context engineering?

Anthropic's engineering team defined it in September 2025 as the set of strategies for curating and maintaining the optimal set of tokens during inference. Tokens are the small pieces of text a model reads, and inference is the moment it runs. The prompt is one part of that set. The rest is everything else the model reads on a given step: tool definitions, retrieved documents, records from your systems, the conversation so far and any notes the agent wrote to itself.

Anthropic describes a model as having a limited attention budget, and every token uses some of it. So the goal is the smallest set of useful tokens that gets the step right. For an enterprise agent, that means a decision for every source. You decide which systems the agent may read and how a record is translated before the model sees it. You also decide when it is fetched, how long it stays, and who may see it.

Context engineering vs prompt engineering

Prompt engineering is about how the instructions are written. Context engineering is about what the model receives alongside them, and it is repeated on every step of an agent's run rather than written once.

AspectPrompt engineeringContext engineering
Works onThe wording of instructionsEverything in the context window on each step
DoneOnce, when the prompt is writtenOn every step, as the agent runs
DataPasted in or describedFetched, filtered and formatted while the agent runs
Fails asA misread instructionStale, conflicting or excess material in the window
Owned byWhoever writes the promptThe team that owns the data and the pipeline

Good prompts still matter. But a precise instruction cannot rescue an agent that is reading last year's policy, and on enterprise data that failure is the more common one.

Why do enterprise AI agents fail? Four context failures

In June 2025 Drew Breunig gave the four failure modes names that have stuck: poisoning, distraction, confusion and clash. Each has a documented case. Google's Gemini 2.5 technical report describes an agent playing Pokémon whose goals and summaries were poisoned with wrong information about the game. It spent many hours chasing a goal it could not complete. The 39% drop over several messages, described above, is clash: early guesses stay in the context and contradict what arrives later. Pick the symptom you are seeing to find the failure behind it.

Diagnose the failure
What is your agent doing wrong?
One wrong fact keeps coming back

Context poisoning

An error enters the context and every later step builds on it.
On enterprise dataA procurement agent misreads a supplier price from an archived quote in step two and writes it into its working notes. Steps three to six check the budget, draft the order and summarize the deal against the wrong price, and each step makes the error look more settled.
The fixCheck figures against the source before they enter memory, and restart from a validated summary when a mistake is found.
It ignores its instructions on long tasks

Context distraction

The context grows so long that the model leans on its own history over its instructions.
On enterprise dataA service desk agent is forty turns into one case. The refund rule sits in the first message, and the agent repeats what it did earlier in the conversation instead of applying the rule to the new request.
The fixCompact long histories into a short summary, and fetch the rule at the moment the request needs it.
It picks the wrong tool or the wrong table

Context confusion

Material the task does not need pulls the model toward a worse answer.
On enterprise dataAn agent receives forty tool definitions and the full CRM schema for every request. Asked for a renewal date, it queries a deprecated contracts table because the column name looked right.
The fixGive each task only the tools and records it needs, translated into business terms.
Two sources disagree and it picks one

Context clash

Information in the context contradicts other information already there.
On enterprise dataRetrieval returns the 2024 and the 2026 discount policy for the same question. The agent answers with the older ceiling, and nobody can tell from the answer which policy it used.
The fixVersion every record, retrieve only the current one, and log which version each answer used.
Failure names from Drew Breunig, How Contexts Fail (June 2025). Examples are illustrative.

A better model or a stronger prompt fixes none of the four, because each one is a problem with what reached the model. So start any diagnosis with the agent's context. Check what was in the window on the step that went wrong, and where each piece came from.

Why a bigger context window makes it worse

Larger windows tempt teams into a simple fix: if the agent is missing something, give it everything. Chroma's study shows the cost. Adding one distractor, a passage similar to the answer but wrong, lowered accuracy compared with the clean test. Four distractors lowered it further. An enterprise search index is full of near-duplicates of exactly that kind: outdated policies, draft contracts, the same process described by three teams.

Size also costs money on every call. You pay for each token in the window, and each one adds delay. An agent that carries a 38,000-token policy manual to answer a question about one discount rule pays for the whole manual every time. Switch between the two windows below to see the same request built both ways.

One request, two context windows
What the model reads before it answers
Sales rep asks the agentCan we offer Acme 15% on their renewal?
Everything in the prompt
100k tokens
  • Instructions
    2k
  • Full pricing policy manual, every versionClash: the 2024 and 2026 ceilings are both here
    38k
  • Raw CRM schema, 400 columnsConfusion: three tables look like contracts
    12k
  • 40 tool definitionsConfusion: 37 tools this request never needs
    9k
  • Last 60 chat turnsDistraction: the history outweighs the rule
    25k
  • Top 20 similar passagesPoisoning risk: an archived quote ranks third
    14k
All four failure modes have material to work with. The answer depends on which of two policies the model weighs more, and the log cannot say which.
Engineered context
4.1k tokens
  • InstructionsWhat the agent does and when it calls the rules engine
    1.5k
  • Discount rule DR-12, version 7Gold tier ceiling, fetched for this request
    0.3k
  • Acme account in business termsTier, renewal date, open opportunity, owner
    0.8k
  • 3 tool definitionsAccount lookup, rules check, approval request
    0.9k
  • Case summaryCompacted from earlier turns
    0.6k
One current rule, one account, three tools. The rules engine returns the ceiling, the model writes the reply around it, and the log records rule DR-12 version 7.
Illustrative token counts on a 100k scale. Yellow is context fetched for this request, orange is context that can mislead the model.

The engineered window is smaller because each piece was chosen for this request. The rule comes from a store that tracks each version. The account arrives in business terms, not column names. The decision itself goes to a rules engine that returns a fixed answer. Our post on mapping business rules into the context layer shows how those rule records are written and owned.

Context engineering techniques that work in production

Seven techniques cover the four failures between them. Four of them appear in Anthropic's guidance, and the other three are how our context layer handles rules and permissions.

  1. Retrieve just in time. The agent keeps references, such as a record ID or a stored query, and loads the data when the step needs it. Anthropic describes Claude Code working this way.
  2. Translate before you inject. A raw database schema makes the model guess. Give it labelled business terms (account, tier, renewal date) and keep table names out of the window.
  3. Scope by task and by permission. Each step gets only the tools and records it needs. They are checked against the asker's access before anything is fetched, so records outside that person's role never enter the window.
  4. Version and rank sources. Every rule and document carries a version and an owner. The agent gets the current version, and the log records which version each answer used.
  5. Compact long runs. When a task runs long, the agent summarizes the history and continues from the summary. Anthropic recommends first making sure the summary keeps everything that matters, then trimming, starting with old tool results.
  6. Write notes outside the window. A to-do list or notes file kept in memory lets a long task keep its place after the history is summarized, without carrying every turn.
  7. Split work across sub-agents. A sub-agent works in a clean window and hands back a short summary. Anthropic puts that summary at 1,000 to 2,000 tokens, so the lead agent never carries the raw material.

Two of our builds show what this looks like in practice. An insurance client's daily MI alerting system calculates every figure in PostgreSQL views. The model gets finished numbers and writes a sentence around them, and a check step flags any figure the model was not given. Pinkcube's chat agent answers from a knowledge base built from support conversations the Pinkcube team had already resolved, checked before the agent can use them. After 76 hostile test scenarios, the worst-scoring case became a rule. Questions about a specific product now go straight to a support ticket and skip the knowledge base search (Pinkcube case study).

Where a context layer fits

One agent can carry its own code for fetching data. Ten agents built that way each keep their own copy of the rules, their own data translations and their own access filters, and over time they stop matching. A context layer moves that work into one shared system. Sources are connected once and access is checked before anything is fetched. Rules are stored as records with versions, and every fetch is logged with where the data came from.

Our enterprise context layer is built inside your private cloud network (VPC), on n8n and the Model Context Protocol. MCP is the open standard AI tools use to connect to company data, and every agent reads from the layer through it. The five-layer blueprint covers how each layer is built. Our guide to context layer vs semantic layer covers how it differs from the metric definitions your reporting tools already use. Our explainer what is an MCP server covers the protocol itself.

How to start engineering context for one agent

  1. Pick one workflow and log the full context window on every step of a sample of real runs.
  2. For each wrong answer, name the failure: poisoned, distracted, confused or clashing.
  3. List every source the agent reads, with its owner and how stale it can get before an answer goes wrong.
  4. Replace raw database schemas with business terms, and pasted policies with rule records that carry versions.
  5. Cut the tool list to what each step needs, and check permissions before retrieval.
  6. Decide when long runs get summarized, and check figures before they enter memory.
  7. Rerun the same sample and compare answers, tokens and latency against the first log.

The same log answers the governance questions later: who asked, what the agent read and which version of each rule it applied. Our post on data governance for AI covers what to record, and AI guardrails covers the checks that stop a bad answer before it becomes an action. Our comparison of AI audit logging tools shows which ones record what an agent read, returned and changed.

Frequently asked questions

What is context engineering in AI?

Context engineering is designing everything an AI model receives when it runs: instructions, retrieved data, tool definitions, conversation history and memory. Anthropic defines it as curating the optimal set of tokens for each step. For agents, it happens again on every step of a run.

What is the difference between context engineering and prompt engineering?

Prompt engineering shapes how instructions are worded. Context engineering decides what data, tools and history sit beside those instructions, how they are formatted, and when they are fetched or dropped. Prompt engineering is one part of context engineering.

Why do AI agents fail in production?

When the model can do the task, the failure usually sits in the context. The agent may keep building on an early error, or carry a history so long it crowds out the instructions. It may get tools and records the task did not need, or sources that contradict each other. Logging the full context on the failing step usually shows which one.

What is context rot?

Context rot is Chroma's name for the decline in model reliability as the input grows. Its July 2025 study of 18 models found that performance grows increasingly unreliable as the input gets longer. Passages that look like the answer but are wrong lower accuracy further.

Does a larger context window fix agent failures?

No. A larger window lets in more irrelevant or conflicting material. That makes distraction, confusion and clash more likely, and adds cost and delay to every call. A smaller window chosen for the step usually does better.

What does a context engineer do?

A context engineer designs how data reaches the model. That covers which sources an agent may read, how records are translated into business terms and how access is checked before anything is fetched. It also covers when history is summarized and what is logged for each answer.

To find the workflow where better context would pay off first, start with the AI Audit, or book a discovery call and walk through an agent that is failing today.

More on this topic

Put your first workflow on the board

Answer a few questions about your team and what you want automated, then a senior partner scopes it with you on the call.