Move the execution boundary, keep the task

Retrospective edition, scoped to 12 February 2026. The vector examples and browser model are synthetic contract demonstrations, not measurements of a deployed model or cloud endpoint.

The local implementation returns document A. The server implementation returns document B. Both expose a method named SearchAsync, both return three results, and both say they use the same model. The deployment diagram looks correct. The user's task has nevertheless changed somewhere between the two boundaries.

The cause may be less dramatic than a different neural network. One path normalizes a vector and the other does not. One truncates input before tokenization, another after it. The server includes documents the local view was not permitted to search. A stale model artifact sits behind an unchanged configuration label. The interface remains compatible while its meaning drifts.

My starting point is that placement is an implementation decision only after the task contract is explicit. Otherwise “move inference to the server” can quietly mean “change preprocessing, evidence scope and ranking at the same time.” The resulting timing comparison may be perfectly measured and still answer the wrong question.

We will follow one query vector through two paths and identify the conditions under which their results deserve to be compared. The reader should be able to distinguish a numerical tolerance, a ranking disagreement and an authorization error, rather than collapsing them into one vague parity score.

A matching method signature is the weakest agreement

Suppose a toy corpus contains three-dimensional vectors. The query is [1, 1, 0]. Document A is [1, 0, 0]; document B is [1, 1, 0]; document C is [10, 0, 0]. With cosine similarity, B is the exact directional match. With an unnormalized dot product, C can win because its magnitude is much larger.

Neither arithmetic operation is inherently defective. They answer different questions. A system that labels both scores “similarity” without declaring the metric can change behavior while preserving every type in its public API.

The interactive model uses these deliberately small vectors. It does not generate embeddings. You can change the query components, the server's normalization and a simulated axis-order fault. The two ranking paths recompute independently. A separate tenant setting changes which candidate documents are eligible, so the display can distinguish a numerical difference from an evidence-scope violation.

The example's value is inspectability. You can calculate a dot product or norm by hand and challenge the result. It is not a substitute for testing the actual tokenizer, model graph and execution providers of a real application. Those components introduce additional behavior that a three-number model intentionally excludes.

Freeze the artifact, not just the product name

A model name is often a moving reference. A repository can update weights, a converter can change a graph, and an application can alter pooling or normalization without changing the phrase used in a product description. A reproducible boundary comparison needs exact artifact identity and the transformation that interprets its output.

For an ONNX-based implementation, that includes the graph bytes, tokenizer assets, vocabulary, selected inputs and outputs, and any preprocessing or pooling code. A hash identifies bytes; it does not prove that those bytes are appropriate. It does, however, prevent two paths from accidentally comparing different artifacts under one friendly name.

ONNX Runtime 1.22.1 was available before this edition's context date, as established by its dated NuGet catalog record. That is a historical availability fact, not evidence that this article ran that package. The examples below stay at the contract and arithmetic layer; no later runtime experiment is imported into this edition.

A manifest should also identify what can vary intentionally. A CPU and another execution provider may produce slightly different floating-point outputs while preserving useful ranking behavior. A tolerance can admit that difference. It should not accidentally admit a different tokenizer or an unauthorized candidate set.

Compare the stages before comparing the final answer

The first comparison is input identity. Did both paths receive the same text or token sequence? The second is preprocessing: normalization, casing, truncation and special tokens. The third is the model output interpretation. The fourth is candidate selection. Only then does the final ranking become meaningful.

This ordering makes a failure actionable. If token IDs already differ, inspecting the top-three result does not tell you whether the model execution is equivalent. If vectors agree but rankings differ, investigate the candidate set, metric, tie-break rule and filters. If rankings agree on three easy queries, that does not prove the preprocessing contracts agree on long or unusual inputs.

A small C# helper can make the numerical boundary explicit:

static double Cosine(ReadOnlySpan<float> left, ReadOnlySpan<float> right)
{
    if (left.Length != right.Length || left.IsEmpty)
        throw new ArgumentException("Vector dimensions must match.");

    double dot = 0, leftNorm = 0, rightNorm = 0;
    for (int i = 0; i < left.Length; i++)
    {
        if (!float.IsFinite(left[i]) || !float.IsFinite(right[i]))
            throw new ArgumentException("Non-finite vector value.");
        dot += (double)left[i] * right[i];
        leftNorm += (double)left[i] * left[i];
        rightNorm += (double)right[i] * right[i];
    }
    if (leftNorm == 0 || rightNorm == 0)
        throw new ArgumentException("Zero vectors have no direction.");
    return dot / Math.Sqrt(leftNorm * rightNorm);
}

This is illustrative arithmetic using a .NET 9 baseline, not a claim about a production vector-search implementation's performance. Double accumulation reduces some intermediate rounding compared with accumulating in float, but it does not make every high-dimensional comparison exact. The rejection of non-finite and zero vectors is a declared contract, not an incidental exception from a division.

Normalization is not a decorative cleanup step

For the toy query, normalization divides each component by the vector norm. If the same positive scale is applied to all components, cosine direction remains unchanged. Dot-product magnitude does not. If two components are swapped, direction can change even though the vector length remains three and its norm remains the same.

That gives us two distinct counterexamples. Scaling a vector should not change a cosine ranking. Swapping axes should generally change it. A parity check that verifies only dimension and norm will miss the second error. A test that compares only one document ID may miss both when the corpus happens to make the winner robust.

The model lets you make this difference visible. Its cells show the actual vector components before and after each selected transformation. The moving record carries the current vector or candidate boundary through the calculation; scores appear only after candidate filtering and scoring. The motion depicts construction of a result, not the passage of time through a real model.

For real embeddings, a useful numerical report can include maximum absolute component error, a norm comparison and a ranked-result comparison under a fixed candidate set. None of those measurements alone establishes semantic quality. They establish selected forms of implementation agreement.

Authorization belongs before ranking

Now add a Red-tenant document with a very strong score for a Blue-tenant user's query. If the server ranks the whole corpus and filters only the displayed top three afterward, it may return fewer eligible results or reveal information through counts and ordering. If it accidentally omits the filter entirely, a high-quality ranking becomes a data-disclosure problem.

The candidate policy should be shared as a task contract: which tenant, which public documents, which version and which revocations apply? The local and server implementations may enforce it differently, but the eligible set should be independently testable.

A typed request can carry an expected catalog version while trusted application context supplies tenant scope:

public sealed record SearchRequest(string Query, string CatalogVersion, int Limit);
public sealed record SearchScope(string TenantId);
public sealed record Candidate(string Id, string TenantId, bool IsPublic, float[] Vector);

static bool Eligible(Candidate item, SearchScope scope) =>
    item.IsPublic || item.TenantId == scope.TenantId;

The request does not get to choose its own trusted tenant merely because a JSON field exists. The catalog version is also not a permission grant. It identifies the evidence revision the caller expects, while the application decides whether that evidence remains accessible.

In the visual, an intentionally unsafe server-scope toggle can include a foreign document. The display marks the scope mismatch even if the top-ranked document happens to remain the same. This is important: matching outputs on one query can conceal a boundary that is already wrong.

Ties and failures are part of parity

Two candidates can have equal scores under the chosen representation. If one implementation preserves insertion order and another sorts by an unstable container iteration order, the returned list can differ without a meaningful numerical disagreement. Define a deterministic secondary key, such as an ordinal document identity, when stable ordering is part of the interface.

Also compare the failure behavior. What happens to a zero query vector? An unknown catalog version? A request that exceeds the maximum input length? A canceled request? Returning a stale result after a failed server call may make the screen look responsive while violating the task's freshness contract.

The strongest objection is that exact parity can be unnecessary. A product may intentionally accept approximate retrieval or different execution-provider results. That is reasonable when the tolerance is specified in terms of the user's task. It is not a reason to omit the comparison. It is a reason to define which differences are acceptable and which require investigation.

A useful test suite contains both invariants and permitted variation. Tenant eligibility should not vary. A small floating-point difference may. A deterministic tie rule should remain stable when scores are equal by the application's definition. The system can then report a specific contract failure instead of treating every different list as equally serious.

Timing needs the boundary written beside it

Moving inference changes more than model execution time. A browser path may pay model download, initialization, memory and UI scheduling costs. A server path may pay serialization, network, queuing and shared-capacity costs. A local loopback HTTP call is not a cloud network measurement.

Before timing, name the interval: cold startup, warm query, end-to-end interaction, or server execution only. Include or exclude artifact loading deliberately. Keep input preparation and candidate policy consistent so a faster result is not simply a smaller task.

This edition provides no runtime timing table. Its purpose is to establish the contract that would make such a table interpretable. A synthetic arithmetic visualization cannot tell you whether a user's device will initialize a model quickly, whether a service will queue under load or whether a particular model produces useful semantic matches.

Those questions deserve their own bounded experiments. The parity contract makes them easier to design because it identifies what must remain fixed and where the measured interval begins and ends.

The migration is ready when its differences have names

A parity failure should retain the smallest inputs needed to reproduce it without copying unrelated user data. For a controlled fixture, that can be the exact vector, candidate identities, metric and expected ordering. For a real model, the necessary token and artifact evidence needs an explicit access policy. “Save every prompt forever” is neither required by the parity argument nor a substitute for identifying which stage changed. The useful record is the one that lets another engineer challenge the same boundary.

Take a query that exercises truncation, a query that creates a tie, a zero-vector failure and a scope-sensitive case. Compare each stage, not only the final list. Record the artifact identities and the policy that selected the candidates. Then decide whether the remaining differences are numerical tolerance, intentional product behavior or a defect.

The goal is not to force every implementation into identical internals. It is to make the same user request mean the same thing across those internals. Once that is true, placement becomes a useful engineering choice. Until then, the phrase “the same model behind HTTP” is only a hypothesis.

GO FURTHER

Inspect the contemporary laboratory.

This is a newly reconstructed historical edition, not a claim of earlier authorship. Later executed experiments, their exact environments and original results are preserved in a separate notebook.

Open the current laboratory notebook ↗

Download the current reconstruction sources ↗