RAG in .NET: Start with the Retrieval Contract

START WITH THE MECHANISM

Interactive companion and laboratory update: 12 November 2026. The editorial date places the topic on the timeline; this current explanation and its measured evidence retain their actual preparation and execution dates.

THE FULL INVESTIGATION

RAG in .NET: Start with the Retrieval Contract

Dates and technology context

This is a retrospective editorial timeline. Topic context: 10 September 2025. First publicly available: 12 November 2026. Current text, visuals and laboratory revisions were prepared on 12 November 2026.

The later local ONNX implementation uses SDK 10.0.400, runtime 10.0.15 and Microsoft.ML.OnnxRuntime 1.22.1. Its model snapshot, 18 held-out queries and execution results belong to 12 November 2026, not September 2025.

A fluent answer can conceal a broken search system. It can also make a good search system look worse than it is. If we evaluate retrieval-augmented generation only by reading the final response, we lose the ability to distinguish those failures.

Consider a support assistant asked whether a customer may cancel an annual contract. The answer depends on the contract version, jurisdiction, customer category and effective date. A paragraph about cancellation may be highly similar to the question and still be the wrong evidence. Retrieval has succeeded in finding related language while failing to find material that can support the decision.

The engineering problem begins before prompting. We need a retrieval contract: what evidence may enter an answer, how it is selected, what metadata travels with it and what the system does when the evidence is insufficient. The generation layer then operates inside that contract rather than trying to repair it with confident prose.

Write the questions before building the index

Start with a small collection of real question types, not a pile of documents. Include questions that should be answered, questions that need clarification and questions that must remain unanswered. “What does this paragraph say?” is a different task from “Does this policy apply to my situation?”

For each question, identify the smallest set of evidence that would allow a careful human to answer. Record acceptable source versions and any facts the system must not infer. A cancellation question might require both a policy clause and a customer-specific amendment. Retrieving either alone is not sufficient.

Use this material as an evaluation fixture. Keep the expected evidence independently authored, rather than asking the same model that generates answers to decide which sources it should have found. Otherwise the test can become a rehearsal of the model's own assumptions.

Do not make every test answerable. An assistant that always finds something can appear helpful while manufacturing coverage. Deliberately include a policy that is missing, an obsolete document, conflicting versions and a question whose key term is ambiguous. A correct refusal or clarification is a successful outcome when the contract requires it.

Model evidence as more than text

01 / See the idea

Evidence has more than one clock

Enlarge diagram

A chunk needs an identity that survives retrieval, reranking and generation. It also needs a source identity and version. Without those, the final citation may point to a document that has changed since the answer was generated, making the answer impossible to inspect.

The following .NET 9 data model is illustrative and has not been executed. It describes an application contract, not a required Azure SDK type.

public sealed record EvidenceChunk(
    string ChunkId,
    string SourceId,
    string SourceVersion,
    Uri SourceUri,
    string Heading,
    string Text,
    DateTimeOffset RetrievedAt,
    DateTimeOffset? EffectiveFrom,
    DateTimeOffset? EffectiveUntil);

public sealed record RetrievalResult(
    string QueryId,
    IReadOnlyList<EvidenceChunk> Evidence,
    string IndexVersion,
    string RetrievalPolicyVersion);

Publication time, retrieval time and effective time are different concepts. A policy can be published before it becomes effective. A page can be retrieved long after publication. A correction can replace an earlier statement without changing the URL. Preserve those distinctions instead of placing every date in one generic Timestamp field.

Source versions need not mean storing every page forever. They can refer to an authorized snapshot or content hash with a retention policy. What matters is whether an investigator can establish which evidence the system actually saw. That is a stronger requirement than displaying a clickable link.

Treat chunking as a hypothesis about meaning

Chunking is often discussed as a choice of token count and overlap. Those settings matter, but the deeper question is what information must stay together to remain interpretable.

A heading can define the scope of the paragraphs beneath it. A table can contain a footnote that reverses an apparent rule. A code example can depend on a warning immediately above it. If chunking separates these elements, retrieval may return text that is locally plausible and globally misleading.

Start with a structure-aware baseline: preserve document identity, headings and stable locators, and avoid splitting a logical condition from its consequence. Then test whether the resulting chunks are too broad for precise retrieval or too narrow for interpretation. There is no universally optimal chunk size independent of the corpus and questions.

Overlapping chunks create another issue: apparent agreement. Five retrieved passages may be five copies of the same underlying claim. The answer layer should know whether it has independent sources or duplicate fragments. Group evidence by source and version before treating repeated language as corroboration.

Separate relevance from permission

The most relevant document may be one the caller is not allowed to see. Authorization therefore cannot be a final cosmetic filter applied after the answer has been generated. By then, restricted text may already have influenced the response.

Build the search scope from authenticated application context. Never ask the model to provide a tenant identifier or list of allowed groups and then trust those values. The model may express the user's request, but the service determines the user's authority.

Azure AI Search documents a security-filter pattern using principal identifiers in an index field. The filter is a query mechanism; it is not itself an authentication system. The application remains responsible for obtaining trustworthy identity information and applying the correct scope. Azure AI Search security filter pattern.

Test authorization with deliberately attractive forbidden documents. If the inaccessible document is the best semantic match, the pipeline must still exclude it. Test the source retrieval endpoint as well as search, because a user should not be able to bypass filtering by guessing a chunk identifier from another tenant.

Choose retrieval mechanisms for different mistakes

02 / A concrete example

Best match can still be forbidden

Enlarge diagram

Vector retrieval is useful when the question and source express related meaning using different language. It is not a guarantee that the retrieved passage contains the exact identifier, exception or version required by the task. Azure AI Search supports querying numeric representations of content, but similarity remains a retrieval signal rather than a truth judgment. Azure AI Search vector search overview.

Lexical retrieval has a different strength. A product code, exception name or contract identifier can be decisive even when its surrounding text is semantically ordinary. A hybrid strategy can combine lexical and vector candidates. Azure documents hybrid queries and the merging of their ranked results, but the existence of that capability does not establish the best settings for a particular corpus. Azure AI Search hybrid retrieval.

Compare mechanisms against the question fixture. Look at which questions improve and which regress. A higher aggregate retrieval score can conceal the loss of a small, important class of exact-match questions. If those questions concern entitlement or safety, their importance may be much greater than their frequency in the test set.

The historical baseline here assumes Azure AI Search vector capabilities available by the article's technical context. It does not depend on later agentic retrieval features, automatic platform integrations or a current portal workflow. The platform's vector-search availability was announced before this design. Microsoft's Azure AI Search announcement.

Keep the model behind an answer interface

Semantic Kernel can supply the answer-generation adapter, but the application should own the evidence contract. Do not make the rest of the service depend on a provider's particular representation of chat history or tool messages.

A simple boundary makes it possible to test retrieval without making a model call and to test generation against a fixed evidence bundle. This is especially useful when a dependency changes and we need to locate the source of a regression.

// Application-owned interfaces. Illustrative, not executed.
public interface IEvidenceRetriever
{
    Task<RetrievalResult> SearchAsync(
        string question,
        AuthorizedSearchScope scope,
        CancellationToken cancellationToken);
}

public interface IEvidenceAnswerer
{
    Task<GroundedAnswer> AnswerAsync(
        string question,
        RetrievalResult evidence,
        CancellationToken cancellationToken);
}

AuthorizedSearchScope must be created by trusted server code. GroundedAnswer should distinguish an answer, a clarification request and an insufficient-evidence response. The Semantic Kernel implementation belongs behind IEvidenceAnswerer; no particular connector API is implied by this fragment.

The generation instruction should describe the role of the evidence, but it is not a security boundary. Source text can contain instructions, examples of instructions or hostile content. Treat it as quoted data. Do not let retrieved material redefine the caller's permissions, choose new external destinations or silently expand tool access.

Citations need validation after generation

03 / Watch the mechanism

A valid citation can support a wrong claim

Enlarge diagram

Static view. Play to see how the mechanism unfolds.

A model can emit a valid-looking citation identifier that was never retrieved. It can also attach a real identifier to a claim that the passage does not support. The first problem is structurally testable. The second requires semantic evaluation and careful review.

Return citation identifiers in structured output and verify that each belongs to the evidence bundle. Resolve URLs on the server rather than accepting arbitrary URLs generated by the model. This prevents the renderer from turning invented links into an appearance of authority.

// Structural check only. It does not prove semantic support.
var allowed = retrieval.Evidence
    .Select(e => e.ChunkId)
    .ToHashSet(StringComparer.Ordinal);

bool referencesOnlyRetrievedEvidence =
    answer.Citations.All(c => allowed.Contains(c.ChunkId));

Do not name that boolean IsGrounded. It checks membership, not meaning. A paragraph can cite the correct document while reversing its conclusion. Naming a check accurately helps prevent a small validation from being promoted into a much larger guarantee.

For sensitive answers, evaluate claims individually. Ask whether each material claim is supported, contradicted or absent from the retrieved evidence. Preserve uncertainty rather than rewarding an answer merely for sounding complete. Sometimes the best answer is shorter because it refuses to fill a gap.

Measure retrieval and generation independently

At the retrieval layer, track whether the expected evidence appears among the returned candidates and how much unrelated material accompanies it. Also track source-version correctness and permission violations. A perfect semantic match from an obsolete or forbidden source is not a successful retrieval.

At the answer layer, use fixed evidence bundles. Include a complete bundle, an incomplete bundle and a deliberately conflicting bundle for the same question. Observe whether the answer changes appropriately. If the model gives the same confident answer in all three cases, the system is not using evidence as strongly as its interface suggests.

Then run the combined pipeline. Preserve the query, retrieval policy, index version, selected evidence identifiers, model configuration and outcome. Separate a failed search call from a search that returned no admissible evidence. These conditions may look similar to a user but demand different operational responses.

Repeat selected questions with equivalent phrasing. Do not assume that one successful wording establishes robust behavior. Keep a holdout set that was not used to tune chunking or prompts. Otherwise the evaluation can become a memory of the development process instead of a test of the system.

Know when RAG is the wrong interface

Some questions are better answered by a deterministic query, a form or a direct link to an authoritative policy. If the task requires an exact balance, transaction status or entitlement decision, language generation may add ambiguity without adding useful judgment.

RAG is especially useful when a person needs help locating and synthesizing evidence across documents. It is less defensible when the evidence is incomplete, permissions cannot be enforced or the task demands a level of certainty the source material cannot provide. More retrieval does not repair a missing source of truth.

The retrieval contract makes that boundary visible. It says what the system knows, where that knowledge came from and how the application behaves when knowledge runs out. Only after those questions are answerable should we spend serious time making the response sound better. A sentence about a retrieval contract is not yet a working retriever. The implementation appendix below closes a deliberately small part of that gap without claiming a complete answer-generation system.

A later local implementation makes the boundary executable

Implementation note, prepared 12 November 2026. The original September 2025 discussion remains a historical design context. The following model snapshot, runtime configuration and executed results belong to this later laboratory.

The shared laboratory accompanying this article and the evaluation article uses a real sentence-embedding model, executed locally through Microsoft.ML.OnnxRuntime in .NET. It does not return hand-authored vectors, call a remote embedding service, or ask a hosted language model to judge answers. The only network activity needed for preparation is downloading the pinned public model artifacts and software dependencies.

The model is sentence-transformers/all-MiniLM-L6-v2, obtained from its official repository at commit 1110a243fdf4706b3f48f1d95db1a4f5529b4d41. The repository describes a sentence-similarity model with 384-dimensional output and an Apache-2.0 license. Its documented use with raw model output includes attention-mask-aware mean pooling and normalization. Those operations are part of this implementation, not an assumption that every transformer output is already a sentence vector. Official model card.

The repository history contains versions from 2021 and ONNX-related changes before the original article's context. The repository API also references the identical ONNX file hash at commit c9745ed1d9f207416be6d2e6f8de32d1f16199bf, dated 6 March 2025. That establishes availability of those model bytes, not a past execution of this laboratory. The selected tokenizer/configuration snapshot and all execution evidence remain explicitly recorded as November work. Official repository history.

The .NET project uses SDK 10.0.400, runtime 10.0.15 and ONNX Runtime package 1.22.1. These are explicitly later execution choices. The original illustrative interfaces above are not silently reclassified as code that ran in September 2025.

Keep model acquisition separate from inference

The ONNX file is 90,405,214 bytes. Its SHA-256 is checked against the repository's published large-file object identifier, and the download manifest records hashes for the tokenizer, configuration and model-card artifacts as well. The reproduction script downloads exactly those URLs and refuses a byte-count or hash mismatch.

No pickled model weights are loaded. No remote model code is trusted or executed. The model is an ONNX graph consumed by the locally installed runtime. This does not make every possible model artifact harmless, but it keeps this example away from an unnecessary mechanism for executing repository-supplied Python code.

The source download package intentionally excludes the large model file. It includes the pinned manifest and verification script instead. Reproduction therefore makes the artifact acquisition visible, including the exact revision and license reference, rather than hiding a mutable download inside the application startup path.

The corpus is also a versioned artifact. It contains 20 synthetic policy documents and 20 queries, of which 18 are reserved for the reported evaluation and two are development examples. Expected relevant document IDs were written before the first model execution. The documents contain no customer information; their policy statements are invented to create controlled retrieval distinctions.

A hash of the dataset is embedded in the prepared token artifact. The .NET runner verifies that hash, the model hash, and the correspondence between each token record's text and the corpus. These checks do not prove relevance judgments are correct. They establish that the execution used the intended input rather than a silently modified dataset.

Tokenization is an explicit preprocessing boundary

The runnable .NET component does not claim to implement arbitrary text tokenization. A Python preparation script uses the pinned tokenizer JSON with the tokenizers library, produces token IDs and a reference embedding, and writes them to an inspectable JSON artifact. The .NET program reads those IDs and performs its own real ONNX inference.

This is a practical boundary for a small reproducible experiment. It lets the article demonstrate model execution in .NET without substituting a superficially similar tokenizer whose normalization or special-token rules may differ. A production text endpoint would need to move or expose that preprocessing stage deliberately; the current executable is not such an endpoint.

The serialized tokenizer initially enabled padding to a fixed length. A golden check expecting the unpadded encoding exposed that setting. The preparation script now explicitly disables padding for these single-item inputs and sets a maximum length of 256. The raw tokenizer file remains unchanged and hash-verified.

Two small golden cases verify the encoding of hello world and its uppercase equivalent. They are not a comprehensive tokenizer certification. The more important reproduction mechanism is loading the exact tokenizer artifact and recording the configuration applied to it.

Each token record contains input_ids, attention_mask and token_type_ids. The runner validates their lengths before building the tensors. The current corpus is short enough for the chosen limit; the article does not infer behavior for long documents from this small test.

The Python runtime emitted a platform-support warning identifying Windows Server 2022 as unsupported by its validation code. The actual reference inference completed successfully. Both facts are preserved: successful execution is evidence about this run, not a claim that the warning can be ignored for a supported deployment decision.

From token output to a normalized sentence vector

01 / See the idea

One explicit path from text to a vector

Enlarge diagram

The ONNX graph accepts three integer tensors and returns token-level output with shape batch, sequence length and 384 features. In the runner, each input has batch size one. The code reads the actual output dimensions and rejects a shape that does not match that contract.

The C# input construction is visible in the supplied Program.cs:

var inputs = new[] {
    NamedOnnxValue.CreateFromTensor("input_ids",
        new DenseTensor<long>(item.InputIds, [1, length])),
    NamedOnnxValue.CreateFromTensor("attention_mask",
        new DenseTensor<long>(item.AttentionMask, [1, length])),
    NamedOnnxValue.CreateFromTensor("token_type_ids",
        new DenseTensor<long>(item.TokenTypeIds, [1, length]))
};
using var output = session.Run(inputs);

After inference, the application sums token features only where the attention mask is nonzero, divides by the active-token count, and normalizes the resulting vector to unit length. This mean pooling is an aggregation over token embeddings. It is unrelated to reusing memory through an array pool, despite the shared word pooling.

The Python reference uses the same ONNX graph and independently implements the aggregation with array operations. The maximum component difference across the 40 prepared items was approximately 7.38 × 10^-9, inside the declared tolerance of 10^-5. This checks agreement between the two implementations; it does not establish that their shared model is semantically correct.

A separate .NET check appends masked padding positions and verifies that the normalized result remains within tolerance. Another repeats inference for one input. These checks target shape, masking and reproducibility errors that a plausible-looking nearest neighbor might conceal.

The runtime session limits intra-operation and inter-operation execution to one thread and uses sequential execution. The scope is intentionally small and CPU-only. No claim about GPU acceleration, batch throughput or a high-concurrency server follows from it. ONNX Runtime C# inference documentation.

Filter eligible documents before selecting neighbors

02 / See the idea

Similarity does not authorize a candidate

Enlarge diagram

The corpus contains Blue-tenant, Red-tenant and public documents. Several Red documents deliberately resemble Blue documents while specifying different limits or permissions. Their purpose is to make unauthorized similarity visible.

The query fixture supplies a tenant scope. Candidate selection occurs before ranking and before taking the top three:

var allowed = corpus.Documents
    .Where(d => d.Tenant == query.Tenant || d.Tenant == "public")
    .ToArray();

The dense method ranks these candidates by dot product of normalized embeddings. The lexical baseline uses Jaccard similarity over lowercase alphanumeric token sets. Both methods use the same eligible documents and a stable document-ID tie breaker. This lexical method is deliberately simple; it is not labelled BM25 or treated as the strongest possible search baseline.

Across the reported cases, the eligible top-three results contained zero documents outside the query's tenant or public scope. An explicitly separate unfiltered diagnostic contained 17 forbidden result slots across the 18 top-three lists. That is a count of slots, not 17 distinct leaked documents or 17 affected users.

The diagnostic illustrates why similarity cannot grant access. It does not constitute a security audit. There is no authentication service in this fixture, and a real application must derive scope from trusted identity rather than accept a caller's arbitrary tenant label. The code demonstrates where the filter belongs once that scope has been established.

All document embeddings are prepared locally for this synthetic experiment. The query ranking does not send restricted candidate text to an external model, but the example still does not claim to solve the lifecycle of a production multitenant index.

The measured interval starts after model loading

The integration owner executed three fresh .NET processes in the exclusive measurement lane. Each process encoded the same 40 items after constructing its ONNX session. The observed batch times were 347.8772, 329.0327 and 335.7522 milliseconds, giving a median of 335.7522 milliseconds.

That interval includes single-item model execution, mean pooling, normalization and component-wise comparison with the prepared reference vectors. It excludes model download, Python tokenization, session construction, ranking and report serialization. The first inference in the batch is included; this is not a separately established steady-state throughput measurement.

The number therefore should not be presented as a user's retrieval latency or as the speed of a complete RAG application. Twenty of the encoded items are corpus documents, and the remainder are evaluation or development queries. A service would normally prepare document vectors on a different schedule from user queries.

All three processes preserved the same numerical parity and ranking checks. These observations support the reproducibility of this small local execution. They do not establish scaling behavior, a concurrency limit or a comparison with a hosted service. Raw records remain in measurement-run-1.json through measurement-run-3.json, alongside the untimed correctness report.

Retrieval can succeed at three and fail at one

Both methods retrieved all manually required evidence within the top three on this small dataset. That result sounds stronger than it is. Some questions need two documents, and the first result can still be the wrong one for the user's immediate decision.

For the question about the maximum size of one uploaded file, both methods placed the organization-storage document first rather than the required per-file-limit document. The necessary document was still present within the top three. A downstream answerer would need to distinguish capacity per organization from size per file instead of copying the first number it sees.

On a differently phrased numerical question, dense retrieval again favored the storage document. On two negation-family questions, the lexical baseline placed an unrelated or less applicable document first, while dense retrieval placed the expected evidence first. The results support inspecting families and individual cases, not declaring a universal winner.

No answer generator was invoked. We have not shown that a model correctly applies the negative rule, compares the numbers or refuses an unsupported conclusion. We have shown what evidence was selected and where it appeared. The companion evaluation article examines those ranking observations in more detail.

This is the practical reason to start with a retrieval contract. It gives the implementation a narrower question it can actually test. A real embedding model, a stable corpus and an eligible-candidate filter are substantial progress, but they are still inputs to trustworthy answering rather than proof that trustworthy answering has been achieved.

Historical versions and verification boundaries

These pins existed before the context date. Availability and static API checks do not establish a successful build or a production-safe patch level. Execution claims are limited to explicitly identified companion checks.

Component	Version	Available by	Source
.NET SDK	9.0.101	2024-12-03	Primary record
.NET runtime	9.0.0	2024-12-03	Primary record
C#	12.0	2024-12-03	Primary record
Microsoft.SemanticKernel	1.0.1	2023-12-18	Primary record

Download the complete laboratory and execution records.