All chapters
  1. Start hereA model knows only what it can see now
  2. Foundations · 1The context window: tokens, messages and order
  3. Foundations · 2Signals and authority: who is allowed to instruct the model
  4. Foundations · 3The context assembly pipeline: select, rank, deduplicate, pack
  5. Retrieval · 4Retrieval fundamentals: lexical search, embeddings, hybrid search, reranking
  6. Retrieval · 5RAG architecture: chunks, metadata, grounding, citations, freshness
  7. Budget · 6Context budgeting: truncation, dilution, token cost and caching
  8. Budget · 7Compression, summarization and compaction, with loss tests
  9. Agents · 8Coding-agent context: repository maps, instruction files, diffs and tool outputs
  10. Agents · 9Working context, durable memory and checkpoint state
  11. Agents · 10Dynamic context: just-in-time retrieval, selective tool disclosure, delegation
  12. Agents · 11Multi-agent context isolation and scoped handoffs
  13. Trust · 12Context security: injection, poisoning, access controls, tenant boundaries
  14. Trust · 13Measuring context quality: recall, utilization, grounding, ablation
  15. Practice · 14Frameworks and standards: what each one assembles for you
  16. Practice · 15Context in production: versions, drift, refresh, observability, incidents
  17. Practice · 16The insurance lab: one submission, every context decision
  18. ReferenceGlossary
  19. AboutAbout this site and its sources
Agents · 10

Dynamic context: just-in-time retrieval, selective tool disclosure, delegation

Loading everything up front fills the window before the work starts. How an agent fetches what it needs when it needs it, sees only the tools a step calls for, and hands a subtask to a fresh window.

You will understand
Why loading everything up front fails, and what fetching just in time costs instead.
You will be able to
Give each step only the tools it needs, and decide when a subtask deserves its own window.
You will see
A 4,031-token roof report read in a subagent's window, and 120 tokens of it coming back.

The first design the team sketches for the underwriting agent is the obvious one. Before the agent starts on case UW-004 (SUB-1004: the application says the roof is 9 years old, the provider says 22), give it everything the insurer has about such a submission: all 63 intake fields, all 20 policy documents, every past decision. Then the agent cannot miss anything.

The script below was written for this page and run against examples/uw-context/. It counts what "everything" costs for UW-004, next to what the current assembler actually puts in the window. Tokens are stand-in tokens (the example's own counter, not any provider's tokenizer).

"""Dynamic-context page: loading everything up front against fetching just in time, for UW-004.

Written for the page and run against the example's own modules. Stand-in tokens (tokens.py).
"""
import json

from assemble import assemble, chunks, past_decisions
from scenario import TOOL_DEFINITIONS, chunk_item, decision_item, load_case, submission_items
from tokens import count_tokens
from window import CONTEXT_LIMIT, OUTPUT_RESERVE, item_tokens

case = load_case("UW-004")
built = assemble(case)
parts = {p["part"]: p["tokens"] for p in built["window"]["parts"]}

all_fields = sum(item_tokens(i, "labelled") for i in submission_items(case))
all_chunks = sum(item_tokens(chunk_item(c, 0, ""), "labelled") for c in chunks())
all_decisions = sum(item_tokens(decision_item(r, 0), "labelled") for r in past_decisions())

print("UW-004, stand-in tokens; the window holds", CONTEXT_LIMIT - OUTPUT_RESERVE, "for input")
print(f"{'source':34} {'up front':>9} {'just in time':>13}")
print(f"{'submission (15 groups / 6 asked)':34} {all_fields:>9} {parts['submission']:>13}")
print(f"{'policy documents (31 chunks / 5)':34} {all_chunks:>9} {parts['retrieved passages']:>13}")
print(f"{'past decisions (3,000 / 3)':34} {all_decisions:>9} {parts['past decisions']:>13}")
print()
print("tool definitions, stand-in tokens each")
for t in TOOL_DEFINITIONS:
    print(f"  {t['name']:24} {count_tokens(json.dumps(t, separators=(',', ':'))):>4}")
print(f"  {'all five, as sent today':24} {parts['tool definitions']:>4}")
UW-004, stand-in tokens; the window holds 7000 for input
source                              up front  just in time
submission (15 groups / 6 asked)        1109           305
policy documents (31 chunks / 5)        4280           713
past decisions (3,000 / 3)            338531           111

tool definitions, stand-in tokens each
  get_roof_assessment       104
  search_guidance            97
  find_similar_decisions    106
  check_eligibility          94
  record_recommendation     159
  all five, as sent today   560

The past decisions alone are 338,531 tokens, about 48 times the room the stand-in model has for input. Drop them, and the fields and documents still take 5,389 of the 7,000 tokens before the system instructions, the tools or the provider's answer. And "all 20 documents" includes three the agent must not use: another tenant's roof rule, a reinsurance memo the underwriting team may not read, and the superseded 2025.4 roof rule. Everything up front is too much and, worse, partly forbidden.

The window that works holds 1,129 tokens of these three sources. It was not built by loading less of everything. It was built by fetching the right things when the step needed them. This page is about that choice: what to load before the work starts, what to fetch during it, which tools to show at each step, and when to send a subtask to a separate window.

Up front, or just in time#

Loading up front means putting everything the agent might need into the window before the first model call. It is simple and fast: one call, no waiting on tools. It works when the material is small, stable and nearly always relevant, such as the system instructions.

Just-in-time retrieval is the alternative: the window starts with a small core and a set of tools, and the agent fetches an item at the moment a step needs it. For example, the agent does not carry the roof rules into the call; it calls search_guidance("roof age on the application and from the provider differ") when it reaches the roof conflict, and five passages come back.

The idea comes from Anthropic's engineering article on context engineering, read for this page. It describes agents that keep lightweight references (file paths, stored queries, links) and load the data behind them through tools at run time, instead of retrieving everything before the call. The article names the cost too: exploring at run time is slower than reading precomputed data, and an agent without good tools and guidance wastes its window on dead ends.

When the model itself decides what to look for next, step by step, using search tools and what it learned from the last result, that is agentic search. A grep, a file listing, a database query written by the agent: each result tells it where to look next. For example, the agent reads the provider's answer, sees 22 years against the application's 9, and only then searches for the rule about conflicting roof ages.

Up frontJust in time
What enters the windowEverything that might matterA small core, then what each step asks for
Cost per callEvery item, on every callThe core, plus what was fetched
SpeedOne call, no tool round tripsMore calls, each one waits on a tool
What can go wrongOverflow, dilution, forbidden items in viewThe agent does not fetch what it needs
FitsSmall, stable, always-relevant materialLarge, changing or rarely needed material
Before AI. An object-relational mapper offers the same choice: eager loading fetches every related row with the parent, lazy loading fetches a relation the first time code touches it. Eager loading of a large collection fills the heap; lazy loading in a loop causes the N+1 query problem, many small round trips. The comparison holds well. Where it stops: an ORM's lazy load is triggered by code you wrote, deterministically, while an agent decides what to fetch, and it can decide not to.

Most real systems mix the two. The article describes Claude Code that way: its instruction files are loaded up front, and everything else in the repository is found with search tools when needed. The underwriting agent mixes them too. The system instructions and the new turn are always in the window; the rules, the provider's answer and the past decisions are fetched.

The failure of just in time: the agent never looks#

Fetching on demand moves a decision from your code to the model: what to look for, and when. If the model skips a fetch, the window lacks the item, and nothing in the window says so.

Case UW-008 is that failure. Its facts are those of UW-001 (roof 31 years, wind zone C), but the scripted agent skips search_guidance and "remembers" the roof rule. Real output of run.py --case UW-008 in a copy of the example:

UW-008  shared 1   1349   15/9   decline      R-ROOF-01                decline      PASS ungrounded ['R-ROOF-01']
1/1 match the expected recommendation

The recommendation is right and the citation is not grounded: R-ROOF-01 was cited, and its text was never in the window. (The model recommends; a person decides. Here the underwriter would read a citation nobody can check against the window.) The window is smaller (1,349 tokens against about 2,300 for its siblings) because the passages were never fetched. A just-in-time design needs a check that the fetches a step requires did happen; Measuring shows how grounding catches this one.

Common confusion. Just in time is not "retrieve less". The current version of the example retrieves the top 10 passages and keeps 5; what changed from up front is that retrieval is driven by the step's question, filtered for tenant, audience and version before similarity (Retrieval), and recorded in the manifest.

Progressive disclosure#

Progressive disclosure means showing a short summary of what exists first, and the detail only when it is asked for. For example, a table of contents before the chapters: a reader sees twelve chapter titles in twelve lines and opens the one they need.

For an agent, the summary is a list of names and one-line descriptions; the detail is a tool call away. The coding-agent fixture of this site does this with a repository map: 163 stand-in tokens that name every file, instead of 6,131 tokens of file contents. The view below runs the same scripted coding session both ways, side by side, step by step.

StepJust in time (map, then reads)Up front (every file at the start)
1, before any tool call3466,477
16, end of the session8,99613,897
16, with old tool results cleared1,3209,272

(Synthetic stand-in tokens from repo_context.py, with and without --upfront and --clear.) Up front costs 6,131 extra tokens on every call of the session, two thirds of them for a data file the task never opens. Coding agents walks through the session itself.

Claude Code applies the same pattern to skills, according to its documentation read for this page. At the start of a session only each skill's name and description are in the window (the combined description text is cut at 1,536 characters in the listing). The full SKILL.md body enters when the skill is invoked and then stays for the rest of the session, which is why the documentation advises keeping it under 500 lines. Supporting files in the skill's folder load only when the body points to them and the agent reads them. Three levels: a line, a page, the reference behind it.

In our insurance agent. The example's assembler selects the six field groups the recommend step needs and records the other nine as not_requested. A progressive design would go one step further: give the model the fifteen group names, one line each, and a tool to fetch a group. The agent then could open the BUILDING group when a question needs it, and the manifest would show the fetch. This is a design sketch; the example does the selection in the assembler.

Showing only the tools a step needs#

Tools are part of the window. Every tool definition (the name, the description and the input schema the application sends so that the model can call the tool) is text the model reads on every call. In UW-004 the five definitions cost 560 stand-in tokens, 7.0 percent of the limit, more than the submission itself.

Tool disclosure is the choice of which tool definitions a given step can see. For example, a step that gathers evidence sees the four read tools; a step that only reads the window and writes a recommendation sees none.

The table splits the example's five tools by the step that needs them. The numbers come from the output above.

ToolTokensNeeded byAuthority
get_roof_assessment104gathering evidenceread, outside provider
search_guidance97gathering evidenceread
find_similar_decisions106gathering evidenceread
check_eligibility94gathering evidenceread, deterministic code
record_recommendation159storing the result, once, at the endwrite

The gathering steps need the four read tools, 401 tokens. Writing the recommendation needs no tool at all: the answer is JSON in the model's reply. Storing it needs record_recommendation, and that is the one tool that should not be in front of the model while it reads a broker's attachment. A model that cannot see a tool cannot call it. Disclosure is therefore an authority decision as much as a token decision: the write tool belongs to the last step, after the recommendation has been validated, and the harness (the runtime around the model, see Harness Engineering, tools) can call it itself.

Anthropic's article makes the general point: a bloated tool set, with overlapping tools or unclear choices between them, is one of the most common failures, and a minimal set is easier for the agent to choose from and easier to keep small over a long run.

What this is not. How a tool is named, described and typed, and how MCP servers expose tools, is the subject of Tools Engineering. This page asks only which definitions are in the window at a given step.

When the tool catalog is large#

The Claude documentation read for this page describes a product feature for a catalog of hundreds or thousands of tools: the tool search tool. The application sends every definition, marks most of them with defer_loading: true, and adds a search tool in one of two variants (regular expressions, or BM25 over natural-language queries). Deferred tools stay out of the window until the model searches; a search returns up to 5 matching tools by default, and the API expands those references into full definitions. The page gives its own example of why: a typical setup of several MCP servers can take about 55,000 tokens of definitions before any work, and model selection accuracy degrades past 30 to 50 tools.

Its documented shape, with two of the example's tools (not run here; the field names type, name and defer_loading as the documentation shows them, the schemas shortened):

{
  "tools": [
    {"type": "tool_search_tool_bm25_20251119", "name": "tool_search_tool_bm25"},
    {"name": "search_guidance",
     "description": "Search the policy documents. Returns passages with rule ids and metadata.",
     "input_schema": {"type": "object", "properties": {"query": {"type": "string"}}, "required": ["query"]}},
    {"name": "record_recommendation",
     "description": "Store the recommendation for the underwriter. Needs write authorization.",
     "input_schema": {"type": "object", "properties": {"submission_id": {"type": "string"}}, "required": ["submission_id"]},
     "defer_loading": true}
  ]
}

The same documentation also says when not to use it: standard tool calling fits better with fewer than 10 tools, or tools used in every request, or small definitions. The underwriting agent has five tools and 560 tokens of definitions. By that guidance it does not need tool search; it needs its own code to choose the tools per step. Deferral also is not authorization: a deferred record_recommendation can still be found and called. Remove a tool the step must not use.

The documentation groups four approaches by where the pressure comes from, which is a useful map:

Approach (Claude documentation)What it reducesThis site's page
Tool searchTool definitions loaded up frontthis page
Programmatic tool callingRound trips of tool calls and results(not covered)
Prompt cachingThe cost of repeating stable definitionsBudgeting
Context editingOld tool results in the historyCompaction

Delegation: a fresh window for a subtask#

Some subtasks need a lot of reading to produce a little knowledge. Delegation is handing such a subtask to another agent, with its own window, and receiving only the result. The agent that receives it is a subagent: an agent started by another agent for one task, with its own instructions, its own tools and a window that starts empty. For example, "read this 4,000-token roof report and tell me the roof age, its source and the method".

Case UW-202 has exactly that report. The provider returns 4,080 stand-in tokens (as a labelled item; 4,031 of raw text) when the decisive fact, the roof age of 27 years, is in its first lines. The script below measures a delegated read. The subagent is not a model here: it keeps the report's first lines, which is what a correct summary of this report must carry.

"""Dynamic-context page: what a delegated read of the UW-202 roof report costs each window.

The subagent here is not a model: it keeps the report's first two lines, which is what a
summary of this report must carry. Written for the page; stand-in tokens (tokens.py).
"""
from scenario import load_case, roof_result_text
from tokens import count_tokens

report = roof_result_text(load_case("UW-202"), 1)
summary = "\n".join(report.splitlines()[:3])
brief = "Read the roof report of SUB-1202 and return the roof age, its source and the method."

print(f"full report, read by the subagent      {count_tokens(report):>5} tokens (its own window)")
print(f"brief sent to the subagent             {count_tokens(brief):>5} tokens")
print(f"summary returned to the main window    {count_tokens(summary):>5} tokens")
print()
print(summary)
full report, read by the subagent       4031 tokens (its own window)
brief sent to the subagent                25 tokens
summary returned to the main window      120 tokens

{"tool": "get_roof_assessment", "submission_id": "SUB-1202", "attempt": 1, "status": "ok", "roof_age_years": 27, "source": "roof-data provider (stand-in)", "method": "aerial image, synthetic"}

Detailed roof report for SUB-1202 (synthetic). Summary: estimated roof age 27 years.

The main window pays 25 tokens to ask and 120 to read the answer. The 4,031 tokens are read once, in a window that is then thrown away. Compare the alternative the site already showed on Budgeting: keep the report in the main window and trim it to the tools share. The lab below is that alternative. Try the "one flat pool" preset and watch the rule disappear.

Setting (UW-202, 2,800 tokens)PackedRule R-ROOF-01 in windowStand-in recommends
Shares 0.31 / 0.30 / 0.24 / 0.15 (default)report trimmed to 419 of 4,080; one past decision droppedyesdecline (expected)
One flat poolreport trimmed to 1,647; all 5 passages, history and past decisions droppednoapprove
Policy share 0.05, tools 0.40report trimmed to 1,120; 4 of 5 passages droppednoapprove

The agent here is a scripted stand-in, so these results show how the mechanism works, not how good any model is. Trimming works for this report because the provider put the roof age first. Delegation works whatever the order, because the subagent reads the whole report and answers a question. It costs an extra model call and the risk that the summary leaves out what mattered, which is why the brief names the facts to return.

Diagram: What enters the window, and when · loaded first, fetched on demand, delegated, or kept for the last step

Read left to right through one run of the underwriting agent. Only the first column is in every window. The second arrives when a step asks for it. The third is read in another window and comes back as a short answer. The write tool appears only at the end. Numbers are stand-in tokens from the example.

  1. Up front, every call
    1. System instructions: Who the agent is and how to treat data blocks. Small, stable, always needed.
    2. Read tools: Only the tools the gathering steps call; the write tool is not shown yet.
  2. Just in time
    1. Fetched by a step: Field groups, passages and past decisions arrive when the step asks, filtered before similarity.
    2. Names before content: A one-line list of what exists; the detail is one tool call away.
  3. Delegated
    1. Subagent: Reads a large source in a fresh window and returns only the answer to a precise brief.
  4. Last step only
    1. record_recommendation: The write tool, shown or called only after the recommendation is validated.

Claude Code's documentation, read for this page, describes delegation in product form. A subagent runs in its own context window with its own system prompt, its own tool access and its own permissions. It starts fresh: it does not see the main conversation's history, the skills already invoked or the files already read; Claude writes it a task message. Only its result returns to the main conversation, and the documentation warns that many subagents each returning detailed results can still fill it. A tools field restricts what a subagent can call (all available tools are inherited when it is omitted), and the built-in Explore subagent is read-only. The documentation also notes that subagents do not reduce total cost: each spends its own tokens. Anthropic's engineering article on context engineering gives a sense of scale for research tasks: a subagent may use tens of thousands of tokens exploring and return a condensed summary of often 1,000 to 2,000.

For a coding agent. "Find every place the roof threshold is used" is a delegation candidate: a read-only subagent greps and opens files, and the main session receives a list of paths and lines, not the files. "Run the test suite and tell me what failed" is another: the 2,000-token logs of Coding agents stay in the subagent's window. Editing is a poor candidate, because the edit and its reasoning belong in the main session's history.

What the brief should contain, what may come back, and what leaks when a whole transcript is passed instead, is the subject of the next page, Isolation.

For a Principal engineer: latency, caching, determinism and authorityEach dynamic choice moves a cost somewhere else; know where

Latency and cost. Every just-in-time fetch is a model turn plus a tool call. A step that needs four fetches costs four round trips that an up-front window would not. Prefetch in code what every run of a step needs (the submission's requested groups, the roof result) and leave to the model only what depends on what it finds. The example does exactly that: the assembler, not the model, gathers the core evidence for the recommend step.

Caching. A cached prefix (see Budgeting) is invalidated when anything before the cache point changes. Changing the tool list per step changes the prefix. The Claude documentation read here says that deferred tools are excluded from the system-prompt prefix and discovered tools are appended inline in the conversation, so tool search keeps the prefix stable. If you choose tools per step in your own code, keep one stable tool list per kind of step, so that each kind has its own cached prefix, instead of a list computed per call.

Determinism and the manifest. An up-front window is a function of the input and the context version. A just-in-time window also depends on the model's choices, which vary between runs (case UW-010 varies with the same window). Record every fetch in the manifest of the call that used it: the query, the items returned, the items kept. Without that, "what did the model see?" has no answer for a dynamic window.

Authority. Tool disclosure is the cheapest control you have, because it removes the possibility of a call instead of judging it. It is not sufficient: a read tool can still carry injected text into the window (UW-005), and a subagent with the same tools as its parent has the same reach. Give a subagent the narrowest tool list its task allows, and keep write tools out of any window that reads external text. Security continues this; the enforcement of permissions belongs to Guardrails Engineering.

When not to delegate. Delegation trades one long window for several short ones plus a summary step that can lose facts. It suits reading-heavy, separable subtasks with a crisp question. It suits poorly a subtask that needs the parent's reasoning so far, or one whose output is long anyway.

A checklist for dynamic context#

CheckWhy
Only small, stable, always-needed items are loaded up frontEverything else costs tokens on every call (6,131 extra per call in the fixture)
Each just-in-time fetch is driven by the step's question and filtered before similarityUp front loaded another tenant's rule, an unauthorized memo and a superseded version
A required fetch that did not happen is detectedUW-008 cited a rule that was never fetched
Each step sees only the tools it needs; write tools only at the step that writesA tool the model cannot see, it cannot call
Tool search or deferral is used for large catalogs, not as authorizationA deferred tool can still be discovered and called
Large reads with small answers are delegated, with a brief that names what to return4,031 tokens read elsewhere, 120 returned (UW-202)
Every fetch and every delegated result is recorded in the manifestA dynamic window cannot be explained otherwise

In one breath#

Load up front only what every call needs and what never changes; fetch the rest at the step that needs it, through tools that filter before they rank, and record each fetch. Show each step only the tools it needs, because tool definitions cost tokens on every call and because a model cannot call a tool it cannot see. When a subtask means reading a lot to learn a little, give it to a subagent with a fresh window and a precise brief, and let only the answer come back.

What comes next#

You can now decide what enters one window and when, including handing a subtask to a second window. But two windows raise new questions: what exactly does the second agent receive, what comes back, and what happens when the underwriting agent passes along its whole transcript, contact details and all? The next page, Isolation, is about many agents with many windows, the scoped handoff brief, and the leaks it prevents.