RAG or Broad Context: Testing Two Paths to the Same Authorized Evidence

Prepared on 12 November 2026. This laboratory builds real context packets and sends them through a real Microsoft IChatClient interface implemented by a scripted local receipt provider. It does not compare the answer quality of generative models.

The argument about retrieval versus a large context window often begins too late. One side proposes selecting a few relevant passages. The other proposes giving the model more of the source material. Before comparing the resulting answers, both sides need to establish which material the application is permitted to supply, which version it represents and what happens when the input is too large.

Those are application contracts. They do not disappear when a model accepts more tokens. A large input can contain an obsolete policy, a document belonging to another tenant or a fragment whose essential condition was removed during packing. A narrow input can make the same mistakes while appearing more efficient.

This article implements two context-building strategies under a common contract. Retrieval uses the top-three document identifiers from the earlier executed local embedding experiment. Broad context includes every eligible document in the same synthetic corpus. Both receive the same tenant and requested corpus version. Both either produce a complete serialized packet inside a declared byte limit or refuse to construct one.

The consumer is explicitly scripted. It returns the version and evidence identifiers it actually received through IChatClient, not an answer to the user's question. That boundary is valuable because it allows the experiment to establish context construction and delivery without pretending an encoder-only model can generate an answer.

Start with the shared evidence boundary

01 / See the idea

Two selections, one eligible source universe

Enlarge diagram

The inputs are frozen copies of the M08 retrieval corpus and result and its evaluation report. The model that produced those rankings was executed locally in the earlier laboratory. This application does not rerun it.

The corpus contains twenty synthetic documents assigned to Blue, Red or public scope. The evaluation queries used here are the eighteen Blue-tenant cases. For those queries, fourteen documents are eligible: the Blue documents and the public documents. Red documents remain in the stored fixture so that an attempted cross-tenant selection can be tested and refused.

The first contract is that both strategies use this same eligibility boundary. A comparison in which broad context receives documents that retrieval was forbidden to inspect would be comparing different information rights, not merely different context strategies.

The tenant value in this laboratory is trusted fixture input. There is no identity provider or production authorization service. The code demonstrates how context construction behaves after scope has been established, not how the application authenticates a real user.

The input manifest records SHA-256 values for the dataset, the saved retrieval result and the model manifest. The executable verifies those bytes before using them. This makes the experiment reproducible without silently changing the corpus to improve the preferred strategy's result.

No document in the example is a real customer policy. Its quantities, permissions and wording exist to create controlled distinctions about scope, evidence and packing.

Give both strategies the same application-owned interface

The context builder accepts a tenant, requested version, strategy, selected identifiers and a maximum byte count. It returns a Bundle describing exactly what was constructed:

public sealed record Bundle(
    string Version, string Tenant, string Mode,
    string[] Ids, string Payload, string Hash, int Utf8Bytes);

The version, tenant and evidence identifiers are not merely labels attached after the text has been concatenated. They participate in deciding which documents can enter the packet and are serialized alongside the selected content.

The retrieval strategy preserves the order of the selected identifiers while removing duplicates. The broad strategy orders all eligible identifiers consistently. This makes the representation deterministic for a given strategy and input. Different ordering can matter to a future generator, but no generator is used to evaluate that effect here.

The code rejects a requested version that does not match the builder's current catalog version. It also rejects any selected identifier missing from the eligible set. It does not silently discard the forbidden identifier and pretend the original request was satisfied.

The distinction between refusal and filtering is deliberate. Broad context derives its set from the eligible catalog. Retrieval receives a proposed selection and must establish that every selected item belongs to that catalog. A stale or unauthorized selection is an observable contract failure, not a hidden quality adjustment.

This small interface is a useful place to hold the invariant. The selection algorithm can change while the rule about eligible, versioned input remains stable.

Broad context is not the same as unrestricted context

The phrase give the model everything is misleading unless everything has a defined boundary. In this laboratory it means all documents eligible for the specified tenant in the requested fixture version. It does not mean all documents on disk, all documents in the organization or every item that happens to resemble the question.

This matters because a broader selection can appear to improve quality by using information the narrower system did not have permission to access. Such an improvement would fail the product contract even if the resulting sentence were factually correct.

The builder constructs the eligible map before either strategy selects its packet. An unknown tenant receives only public documents. A deliberate attempt to insert r-upload into a Blue retrieval packet is refused.

These checks protect the fixture's selection boundary, but they are not a complete security evaluation. The document corpus and tenant labels are controlled. The application does not test revoked credentials, cached private material, logging policies or adversarial model output. Those require a larger system and separate evidence.

The important engineering lesson is the order of operations. Decide which sources are eligible before selecting or packing them. A model's relevance judgment cannot create permission, and a large context window cannot justify ignoring the boundary.

The result is a fairer comparison: three selected documents and fourteen broadly included documents are alternative presentations of the same authorized source universe, not different entitlement policies disguised as architecture choices.

A context limit needs a unit and a failure policy

02 / See the idea

No silent truncation at the byte boundary

Enlarge diagram

The fixture measures the UTF-8 byte count of the serialized packet. The count includes the JSON representation supplied to the scripted client, not only the raw document paragraphs. The application then compares that exact count with the caller's declared limit.

This is a byte budget, not a token budget. Encoding.GetByteCount reports the number of bytes produced by an encoding. It does not estimate the token count of a language model, whose tokenizer and message formatting would need their own measurement. Encoding.GetByteCount documentation.

The distinction is central to interpreting the laboratory. We can test exact packet-size behavior without claiming compatibility with a particular model's context capacity. There is no model capacity selected in this application.

If the serialized packet exceeds the limit, construction fails with context-too-large-no-truncation. The builder does not cut a document, drop its final paragraph or remove an exception to squeeze the input into the limit. A test sets the limit to one byte below a known packet size and verifies refusal. Another uses the exact size and verifies acceptance of the same representation hash.

A production system can choose a more sophisticated policy: select smaller complete units, ask a narrower question or route the task elsewhere. What it should not do is silently turn one evidence contract into another while reporting the original task as complete.

The useful failure state makes the next decision explicit. The current packet does not fit. The application must choose a different authorized packet or a different interaction, rather than assuming that arbitrary truncation preserves meaning.

The implementation keeps that rule close to serialization rather than estimating size from document count:

int bytes = Encoding.UTF8.GetByteCount(payload);
if (bytes > maxBytes)
    throw new InvalidOperationException("context-too-large-no-truncation");

The boundary includes identifiers and structural overhead because those bytes are part of the message actually sent. A test using only paragraph lengths would protect a different representation. This is also why a change in serialization deserves a new packet hash even when the visible prose appears unchanged. The application should measure the object it promises to deliver, not a convenient approximation that happens to be smaller.

Treat source identity as part of the serialized packet

The payload contains the catalog version, tenant and document records. The builder computes its representation hash after serialization and records the UTF-8 byte count. This creates a direct relationship between the reported identifiers and the actual bytes passed to the consumer.

The hash is not a proof that the documents are true or that the resulting answer will be correct. It identifies the representation constructed by this implementation. If the serializer, document order or content changes, the representation may change even when the general topic remains the same.

That is useful for comparing runs. An apparently better answer cannot be attributed to a different model if the input packet also changed without notice. The application should preserve enough information to distinguish the intervention from a hidden change in evidence.

In this small fixture, the payload is JSON and every document is short. The design does not solve efficient storage for very large packets or the licensing conditions for retaining external sources. It avoids those additional concerns by using an owned synthetic corpus.

The evidence identifiers remain valuable even without generation. The scripted receipt can confirm which IDs arrived, and a test can compare them with the expected evidence set. This establishes delivery of the declared packet before asking a separate question about whether an intelligent consumer can use it correctly.

That separation makes the experiment easier to debug. A missing source in the receipt is a construction or delivery problem. A later incorrect answer despite the right receipt would require a different explanation.

Use the real interface without pretending to use a model

05 / See the idea

The client acknowledges evidence, not an answer

Enlarge diagram

The application references Microsoft.Extensions.AI.Abstractions version 9.10.2 and implements IChatClient. The package provides the actual shared interface and message types; the implementation behind that interface is our own scripted class. Pinned package.

The client reads the single user message as a context packet and returns a JSON receipt. The receipt's kind is scripted-receipt-not-answer, and its fields identify the version and document IDs received. It does not summarize, reason over or answer a question about the documents.

This is a real contract test at the IChatClient boundary. It verifies that the code constructs messages accepted by the implemented interface and that the consumer observes the intended packet. It is not a model evaluation simply because the interface commonly connects to model providers.

The client honors cancellation before parsing its input. Its streaming method is deliberately unsupported rather than returning a fake stream and suggesting that streaming behavior has been verified. The example needs only the nonstreaming request/response boundary.

Microsoft's interface documentation also makes clear that applications retain responsibility for concerns such as input sizes and other safeguards unless a particular implementation documents them. The abstraction does not supply an automatic security policy for a packet. IChatClient contract.

Using the real interface keeps future integration visible. A later implementation can replace the scripted client with a real provider, but that replacement creates new obligations: token limits, output interpretation, failure behavior and independent answer evaluation. The current checks do not grant those properties in advance.

A changed catalog must invalidate an old request

03 / Watch the mechanism

A new catalog cannot silently serve an old request

Enlarge diagram

Static view. Play to see how the mechanism unfolds.

The update test creates a new local catalog version in memory. It changes the synthetic per-file upload statement from twenty MB to twenty-five MB and labels the catalog fixture-v2. The preserved M08 input remains unchanged on disk.

A request for fixture-v1 against the new builder is rejected as stale. Requests explicitly naming v2 can construct retrieval and broad-context packets. Their hashes differ from the corresponding v1 packets, and the test verifies that the updated text is present while the earlier upload value is absent.

This is a test of catalog and packet invalidation, not of an updated embedding index. The retrieval selection uses the same source identifier in a controlled example. A real updated corpus could change which documents should rank highest, so its index or ranking artifacts would need their own version discipline.

That limitation should not be hidden. Replacing the text behind an old ranking can be safe for a particular identifier but does not prove that the old ranking remains relevant to every query. The application must decide whether the ranking depends on the changed representation and rebuild it when necessary.

The fixture makes one narrower rule executable: an old requested catalog version cannot silently resolve against a new catalog. That prevents a class of stale-context mistakes without pretending to implement a complete cache or index invalidation system.

The next integration should preserve this separation between source identity, source representation, catalog version and selection artifact. Treating all four as one timestamp is convenient until a change requires knowing which assumption became invalid.

The executed packet results

04 / See the idea

Observed packet sizes are not token prices

Enlarge diagram

The correctness run built two packets for each of eighteen saved dense-retrieval queries. Retrieval supplied three documents in each packet; broad context supplied all fourteen eligible documents. Both packet types contained the expected evidence IDs for every query in this small dataset.

The observed packet sizes were:

Strategy	Packets	Documents per packet	Minimum UTF-8 bytes	Median UTF-8 bytes	Maximum UTF-8 bytes
Recorded retrieval selection	18	3	489	505	578
Broad eligible context	18	14	2,218	2,218	2,218

These figures describe the exact serialized fixture representation. They are not token counts, model charges or a prediction of response quality. The broad packet is constant because the eligible catalog and ordering are constant for these Blue-tenant queries.

The complete run passed 121 assertions. Three verify the copied input hashes. For each of the thirty-six packets, the harness checks eligibility, presence of required evidence and the scripted receipt. The remaining checks exercise exact budget acceptance, over-budget refusal, a forbidden source, public-only behavior for an unknown tenant, stale-version rejection, both updated packet hashes and content, and cancellation.

The test does not claim that broad context improves an answer merely because it contains more documents. Nor does it claim that retrieval is superior merely because its packet is smaller. Both passed the evidence-presence contract on this fixture; their effects on a real generator remain unknown.

The saved observations contain the packet version, hash, document count, byte count and an explicit false value for answer_quality_measured. That field is a small protection against a report later being reused as if the missing experiment had happened.

Compare the work performed before comparing the architectures

A useful production comparison would include the work required to prepare and maintain each input strategy. Retrieval may require extraction, embedding and index maintenance. Broad context may require selecting, organizing and transporting a larger authorized collection. The actual costs depend on the workload and implementation.

The present experiment does not measure those complete preparation paths. It reuses recorded rankings and constructs packets from an in-memory catalog. The embedding model's earlier runtime is not added to the new packet-building time as though the two were one contemporaneous end-to-end measurement.

This matters because the schedule of work differs. Document embeddings may be prepared when the corpus changes, while query context is constructed for each request. A single total without an allocation rule can make either approach appear cheaper by moving shared work outside the measured interval.

A complete comparison should therefore state the corpus-change frequency, query volume, permitted access scopes and acceptance conditions. It should report preparation separately from query execution before explaining how the two are combined for a particular operating decision.

No such operating scenario is assumed here. The bounded result is that two application-owned strategies can construct and deliver explicit packets under the same version and eligibility contract. That is a real mechanism, but it is not the whole economic or quality decision.

A small timing result with a deliberately small meaning

The integration owner ran three fresh processes in the exclusive measurement lane. Each process performed the correctness checks, then timed one thousand alternating operations: a retrieval packet followed by a broad packet, each sent through the scripted client.

The batch times were 52.7129, 52.5871 and 52.1977 milliseconds. Their median is 52.5871 milliseconds. Dividing by one thousand gives a process-average value of approximately 0.0526 milliseconds per packet-and-receipt operation, but that arithmetic must not be turned into a model latency claim.

The timed interval includes selection from the small catalog, JSON serialization, UTF-8 counting, hashing, message construction and parsing the scripted receipt input. It includes both strategies in one alternating batch rather than measuring them independently. Consequently, the result cannot identify which strategy contributed a particular portion of the time.

There is no network request, model generation, tokenization or embedding computation in that interval. There is no large index, concurrent request load or production authorization service. The result describes local code that prepares and acknowledges a context packet.

This narrow measurement is still useful as implementation evidence. It confirms that the tested path can be executed repeatedly and that the reported workload is concrete. It also keeps the temptation to assign most application latency to the model from obscuring the work the surrounding application performs.

The experiment is not a statistical performance study. Three process batches on a shared developer machine expose some variation, but they do not establish a latency percentile, a service-level objective or a general throughput limit. The raw files retain all three observations and the scope statement.

If the next decision concerns comparing the strategies' own preparation cost, the next benchmark should separate them and include representative catalog sizes. If it concerns user experience, the next experiment needs the full interaction and the actual response provider. Those are different questions and should remain different measurements.

The strongest counterargument is that this is not yet RAG versus long context

That objection is correct if the phrase means a comparison of generated answers from two live architectures. The current laboratory does not perform that comparison, and its title describes testing paths to evidence rather than announcing an answer-quality winner.

The value of the narrower work is that the paths themselves can be wrong before a model sees them. A missing permission, stale version or silently truncated exception can invalidate a later quality comparison. Adding a capable generator would make those errors harder to diagnose if the packet boundary remained implicit.

The application therefore tests the part it can specify exactly. It knows which source IDs are eligible, which catalog version was requested and how many bytes its own serialization produces. It does not know which answer a real model would generate, so it does not manufacture that outcome through the scripted receipt.

A critic could also argue that refusing an oversized broad packet is less useful than selecting a smaller one. That may be true for a production interaction. Here, refusal is the explicit baseline that prevents an incomplete packet from being reported as the intended one. A future selection policy can improve usability while retaining that visibility.

Similarly, the stable ID-based update fixture is not an updated retrieval index. Its purpose is to show that a stale requested version cannot silently resolve against new content. Re-embedding, reranking and evaluating the new selection remain separate work.

These limits make the implementation suitable as a foundation, not a finished product. Its contribution is to reduce the number of hidden changes in the later comparison. When a real generator is added, the team can inspect which packet it received rather than guessing whether a quality difference came from the model or from the evidence it was allowed to see.

Where a real answer evaluation would begin

The next step is not simply replacing the scripted client and declaring the comparison complete. A real generator needs a defined answer contract, an independently reviewed acceptance set and a way to preserve failures, clarifications and unsupported claims.

The structured-oracle laboratory offers one candidate layer for narrow numeric and permission propositions. It can check declared facts after they have been represented in a structured form. It does not prove that a generator will extract those facts correctly from either packet strategy.

A fair answer experiment should keep the authorized corpus, question set and expected outcomes stable while changing the context strategy. It should preserve the actual packet supplied to the model and identify any provider-side limitation that affects what is used. If one strategy exceeds the model's input limit, the handling policy becomes part of the comparison rather than an implementation detail to conceal.

Questions requiring an exception or a distinction between scopes deserve special attention. Having the correct paragraph somewhere in a large input does not establish that the answer will apply it. Conversely, a short retrieved packet can omit the very condition that makes a conclusion valid.

The current laboratory leaves those outcomes unknown. That is the correct state of the evidence, not an invitation to infer the preferred answer from packet size. The architecture decision should follow a completed evaluation of the intended task, not a slogan about the end of retrieval or the inevitable superiority of more context.

Source code and execution evidence

Download the complete lab, source record and raw execution results. This package supports the specific execution claims in this article. It is not a production certification.