Vector Search Under Pressure: Distance, Candidates and Index Updates

START WITH THE MECHANISM

Interactive companion and laboratory update: 12 November 2026. The editorial date places the topic on the timeline; this current explanation and its measured evidence retain their actual preparation and execution dates.

THE FULL INVESTIGATION

Vector Search Under Pressure: Distance, Candidates and Index Updates

Dates and technology context

This is a retrospective editorial timeline. Topic context: 7 May 2026. First publicly available: 12 November 2026. Current text, visuals and laboratory revisions were prepared on 12 November 2026.

The preserved M08 vectors, 20-document corpus, exact/approximate results, radius observations and shared .NET10 lab were produced or reused on 2026-11-12.

Prepared on 12 November 2026. The reference compares exact search with a small, materialized sign-bit bucket index over preserved real sentence embeddings. The approximate method is an educational coarse preselection algorithm, not HNSW, a trained quantizer or a hosted search engine.

When a vector search result is wrong, changing the distance function can feel like the obvious response. But the intended document may never have reached the scoring stage. A filter, stale index entry or narrow candidate selection can remove it before the metric has any opportunity to rank it.

Those mechanisms require different repairs. A better similarity function cannot recover a document that was excluded from the candidate set. A wider candidate set cannot correct an outdated representation. A current embedding does not grant permission to retrieve another tenant's material.

This article separates those questions in an executable experiment. It uses the real normalized vectors from the accepted M08 laboratory, exact dot-product search as a mathematical reference, and a deliberately simple approximate index whose candidate choices can be inspected. The same query and eligible corpus are held constant while the candidate radius changes.

The purpose is not to compete with a production vector database. It is to show where quality can disappear, how to measure that disappearance separately from relevance, and why update and deletion behavior belong in the index's contract rather than in a footnote after the benchmark.

Preserve the representation before comparing search

The input consists of twenty synthetic documents and their real 384-dimensional sentence vectors from the earlier local ONNX run. The query vectors come from the same accepted reference artifact. No new model inference is performed in this experiment.

This matters to the comparison. If the exact and approximate methods used different embeddings or different source versions, a change in results could not be attributed cleanly to candidate selection.

The input manifest verifies the copied files against their hashes. Document text, query text and relevance IDs remain unchanged. The exact search and coarse index therefore operate on the same recorded representation.

The corpus is small and intentionally synthetic. It contains related policy statements with differences in units, scope and permission. Its usefulness is inspectability, not statistical representativeness. A result on these twenty vectors cannot establish quality for a large document collection, another language or a different embedding model.

The experiment is therefore a mechanism probe rather than a product leaderboard. Its exact baseline is cheap enough to compute for every query, which lets us inspect where the coarse gate loses a neighbor. At larger scale, preserving a trustworthy reference sample becomes a separate design decision; the difficulty of computing exact results should not be used as permission to redefine approximate success without an independent comparison.

The experiment also avoids generating random vectors and presenting them as semantic embeddings. The only manually specified geometric vectors are tiny arithmetic fixtures used to check normalization and orthogonality. They are not included in the semantic retrieval results.

Keeping those sources of data separate allows the report to answer two different questions honestly: does the mathematical implementation behave on known geometry, and what does candidate selection do to the saved semantic queries?

Exact search is a reference for neighbors, not for truth

01 / See the idea

Exact neighbors and relevant evidence differ

Enlarge diagram

The exact baseline filters eligible documents, computes the dot product with every remaining vector, sorts by descending score and uses a stable identifier as a tie breaker. It then returns the first three entries.

For vectors normalized to unit length, the dot product corresponds to cosine similarity. The implementation checks a known unit-vector case and an orthogonal pair before interpreting the semantic results. These arithmetic fixtures do not establish that cosine similarity is the best measure for every task; they establish the calculation being used.

The code's exact method is compact:

input.Where(e => e.Tenant == tenant || e.Tenant == "public")
    .OrderByDescending(e => Dot(query, e.Vector))
    .ThenBy(e => e.Id, StringComparer.Ordinal)
    .Take(k)
    .ToArray();

Calling this a ground truth needs qualification. It is the exact nearest-neighbor result under the declared representation, metric, eligibility rule and tie breaker. It is not a ground truth about which document best answers the user's question.

The separate manually expected source IDs provide the relevance reference for this corpus. A method can preserve exact neighbors while those neighbors are imperfect evidence for a question. Conversely, an approximate method could omit an exact neighbor and still retain the manually required source.

The report therefore keeps exact-neighbor recall and relevant-evidence recall separate. Combining them into one score would hide whether a failure began in the approximation or in the representation's relationship to the user need.

The stable tie breaker is part of this reference as well. Two scores that compare equally still need a reproducible ordering if the top-three identity set is used as an expected result. A different tie policy can change membership at the cutoff without changing the underlying similarity calculation. The report should not diagnose that as a semantic improvement or regression until the tie boundary is understood.

This is particularly relevant when an update changes chunk identifiers while preserving source text. Source-level relevance and chunk-level neighbor identity answer different questions. The experiment retains both so that a new identifier cannot silently make a refreshed source look like missing evidence, or make an old chunk appear current merely because its source name remained familiar.

Build a real coarse index whose decisions are visible

02 / Watch the mechanism

A coarse gate acts before exact scoring

Enlarge diagram

Static view. Play to see how the mechanism unfolds.

The educational index uses the signs of the first four vector components as a four-bit key. Each document is placed in a materialized bucket when the index is built or updated. At query time, the query receives the same key, and the index selects buckets within a chosen Hamming radius.

Radius zero searches only the bucket with an identical key. A larger radius admits keys differing in more bit positions. Radius four admits every possible four-bit key, so candidate preselection no longer excludes a document because of its bucket.

The selected candidates are filtered by tenant and then reranked with the same exact dot product used by the baseline. The approximation lies in the candidate gate, not in a different final scoring formula.

This is a real index in the limited sense that its buckets are constructed ahead of queries and consulted during lookup. It is not HNSW, which uses a different graph-based mechanism, and it is not a trained coarse quantizer. The first-four-component sign rule is deliberately simple and carries no claim of optimality.

That simplicity has educational value. A reader can identify exactly why a document was excluded: its key was outside the selected radius. There is no need to infer an invisible graph traversal from a final score.

It also creates a clear limitation. Arbitrary coordinate signs may partition the embedding space poorly for the workload. The experiment is designed to observe the consequences of a controllable gate, not to recommend this particular partition as a production strategy.

Keep candidate selection separate from scoring

The coarse search method first identifies matching buckets, then gathers eligible entries and finally calls the exact reranker. It reports both the returned hits and the number of candidates scored.

Those two outputs describe different costs. Candidate count indicates how much final vector scoring the gate permitted. It does not include every operation involved in bucket lookup, sorting, allocation or building the index. A smaller count can therefore coexist with little or no elapsed-time improvement in a tiny corpus.

The implementation enumerates the existing bucket keys and compares their bit distance with the query key. With four bits, the number of possible buckets is small. That design would need re-examination for a substantially different partitioning scheme.

The final score remains a dot product. If a required document is absent because its bucket was not selected, changing the score of the remaining candidates cannot recover it. The next intervention should widen or change candidate selection, or use a different index mechanism.

If the document is present but ranked poorly, the question moves to representation, metric, reranking or the task's relevance definition. The correct repair depends on identifying that boundary rather than treating every miss as a generic vector-search failure.

This separation is the central contribution of the experiment. It turns a disappointing top-three result into a more specific observation about the stage that removed or misordered the evidence.

Eligibility is applied before final ranking

The fixture contains Blue, Red and public documents. Query scope determines which entries may be considered. Both exact and approximate paths preserve that rule.

In the approximate path, matching a bucket does not grant access. The selected bucket entries are still filtered before scoring and returning results. A Red document can be geometrically close to a Blue query and remain ineligible.

This is a fixture-level eligibility test, not a complete authorization system. The caller supplies a trusted tenant label in the test. There is no real authentication provider, document permission service or audit of browser delivery. Those concerns require additional implementation boundaries.

The important point is that both comparison paths share the same rule. Otherwise an apparent quality improvement could come from searching a larger, unauthorized corpus rather than from a better index.

Filtering can also interact with approximation. A narrow bucket may contain several geometrically close but ineligible entries and few permitted ones. The final result can contain fewer than three documents even though more eligible documents exist elsewhere. The index should report that outcome rather than silently interpret an empty result as proof that no relevant source exists.

The observed candidate count helps expose this distinction. No candidates in the chosen buckets is different from no eligible documents in the corpus and different again from no evidence supporting the user's question.

Update the materialized index, not only the source label

04 / See the idea

A changed index invalidates a stale generation

Enlarge diagram

The index stores entries by identity and rebuilds its bucket map when a source is replaced or deleted. Replacing a source removes its older entries before adding the new entry. A replacement with a lower source version is refused.

This is deliberately simple update machinery. It favors inspectability over incremental-index performance. A production index might update affected buckets or graph links instead, but it would still need to preserve the same relationship between current source state and searchable entries.

The fixture tests a source deletion and confirms that the removed source cannot appear in the materialized bucket result. It then adds a new source version and verifies that an older version cannot replace it.

The index also exposes a generation counter. A query prepared against an older generation is rejected after the index changes. This is not a distributed consistency protocol; it is a local guard that makes stale snapshot use visible.

That guard is useful because a caller may hold candidate assumptions or references created before the update. Silently mixing those assumptions with a newer index can produce results whose provenance is difficult to explain.

The laboratory does not simulate a background update racing with arbitrary concurrent readers. Its mutation tests are deterministic. A concurrent production design would need an atomic publication or snapshot strategy and tests at that boundary.

Deletion and missing embeddings are different states

The shared ingestion store distinguishes a deleted source from a new source version whose text lacks a verified embedding. Deletion removes the source's chunks. A changed paragraph can remain stored while being excluded from vector retrieval until its semantic representation is prepared.

The vector index should not conceal that difference. A source absent from searchable entries may be deleted, unauthorized, not yet embedded or outside the selected candidate buckets. Those explanations imply different actions.

For a stale embedding, the solution is not simply to widen the candidate radius. The representation no longer matches the text. For a deleted source, rebuilding an old index from stale input could resurrect content that should remain unavailable. For an unauthorized source, neither re-embedding nor reranking grants permission.

The fixture's exact text-hash lookup prevents newly changed text from borrowing an older paragraph's vector. The index experiment begins only after the original snapshot has been explicitly restored, so exact and approximate results remain comparable.

This is a reason to keep operational metadata close to quality observations. A relevance score detached from source version and readiness can look stable while the evidence underneath it has changed.

The experiment's update tests do not establish a general deletion guarantee across backups, logs or external stores. They establish that the local searchable structures examined here remove or replace the intended entries.

Measure two recalls, and keep their denominators visible

For each of eighteen queries and four radii, the runner records the exact top-three source IDs, approximate source IDs, eligible candidate count, exact-neighbor recall and relevant-evidence recall. This produces seventy-two comparison records.

Exact-neighbor recall is the number of approximate hits also present in the exact top three, divided by three. It answers how much of the declared exact neighbor set survived candidate preselection.

Relevant-evidence recall is the fraction of manually expected source IDs found among the approximate hits. Some questions require more than one source, so their denominator differs from a simple hit-or-miss indicator.

Neither metric measures generated-answer quality. No answer generator is invoked. A retrieved document can contain the necessary rule while a future consumer still applies it incorrectly.

The distinction also explains why perfect agreement with exact search is not the end of evaluation. At radius four, the candidate gate admits all buckets. If the implementation is correct, the remaining behavior should align with the exact baseline. Any remaining relevance limitation belongs to another stage or to the task definition, not to the four-bit preselection.

The recorded query families make this easier to inspect. Numbers, negation and scope can respond differently to a selection policy, and an overall mean can conceal the particular class of question the product cares about most.

Small geometry checks protect the interpretation

Before reading the semantic results, the harness normalizes a hand-specified vector and checks its self-dot-product against one within a small numerical tolerance. It also checks that two orthogonal vectors have zero dot product.

These are arithmetic checks, not synthetic semantic examples. Their role is to catch an implementation mistake that could invalidate every later comparison. They are not mixed into the retrieval quality dataset.

The actual sentence vectors are normalized outputs from the earlier accepted model run. Their provenance remains in the input manifest, and the new experiment does not change the model or tokenizer while varying the candidate radius.

This makes the intervention interpretable: the gate changes, while the representation and final score remain fixed. A future experiment comparing metrics should hold the candidate population and normalization contract fixed instead.

That discipline matters because several plausible changes can improve the same result. A larger candidate set, a different embedding, a scope-aware filter or a new relevance judgment can all alter the top three. Without controlled boundaries, the report may identify a real improvement while explaining it incorrectly.

The laboratory therefore uses a small number of explicit operations and retains per-query records, rather than producing a single score whose causal story is impossible to reconstruct.

The timing boundary is candidate lookup plus exact reranking

The optional measurement mode performs one hundred repetitions of the eighteen queries for each radius. Each radius therefore has 1,800 measured lookups in a process. The index is already materialized before those intervals begin.

The timed path includes bucket selection, eligibility filtering, dot products, ordering and result construction. It excludes model inference, tokenization, index build, source ingestion and network transport. The result is not end-to-end RAG latency.

The runner also records the total number of candidate vectors scored. That gives a mechanism-level observation alongside elapsed time. A smaller radius can reduce candidate work while reducing recall; a larger radius can restore neighbors at additional scoring cost.

On a twenty-document corpus, fixed overhead and ordinary managed allocations can dominate. A result here should not be extrapolated to a million-vector service. The purpose is to expose the quality/candidate tradeoff under a transparent index, not to predict the throughput of an unrelated production engine.

Three fresh processes are sufficient for a bounded local comparison but not a general statistical claim. The integration owner serializes task-owned measurements, while other activity on the developer host remains uncontrolled. Raw batch values should remain available beside any median.

An experiment that needs production-scale behavior should choose a verified implementation, define its indexing parameters and test representative workloads. This educational index provides a way to ask the right questions before that larger comparison begins.

The observed candidate tradeoff

05 / See the idea

Less candidate work was not always faster

Enlarge diagram

03 / See the idea

Wider radius recovered more relevant evidence

Enlarge diagram

The executed results show the expected direction for this particular gate: widening the radius admits more candidates and recovers more of both the exact-neighbor set and the manually expected evidence.

Radius	Mean eligible candidates	Mean exact-neighbor recall@3	Mean relevant-evidence recall@3
0	0.889	0.111	0.194
1	4.444	0.444	0.556
2	9.444	0.741	0.778
4	14.000	1.000	1.000

These are means over eighteen fixed synthetic queries. The two recall columns have different denominators, and neither is a model-answer accuracy rate. The complete per-query records remain available in correctness.json.

At radius zero, some queries have no eligible candidates in the matching bucket. The average candidate count below one is not evidence that the corpus contains less than one relevant document. It describes how restrictive the partition is for these query keys.

At radius four, every bucket is eligible, leaving fourteen tenant-permitted documents per query. The approximate path then becomes an exact rerank over the full eligible population and recovers the baseline set. That is an implementation check, not proof that the four-bit method is a useful large-scale index.

The timing results are less monotonic than the candidate counts. Across three processes on the final shared implementation, median times for 1,800 lookups were 86.3467 milliseconds at radius zero, 286.5602 at radius one, 59.7265 at radius two and 89.8472 at radius four. The order was fixed, and the small managed implementation was not designed to eliminate JIT, allocation or host effects. Earlier runs from before a pipeline-shutdown correction are preserved as superseded and are not combined with these values.

It would be inappropriate to invent a causal explanation for the radius-one result from those totals alone. The program recorded more than final vector multiplications, and no profile attributes the elapsed time to one mechanism. The result does not support a simple statement that fewer candidates were always faster.

The useful conclusion is that the quality loss and scoring-work reduction are observable, while elapsed-time behavior needs its own analysis. If speed were the next decision's central criterion, the experiment should be extended with a controlled warm-up policy, isolated implementation costs and representative corpus sizes, rather than selecting the most favorable number from this table.

Rebuild cost and snapshot validity are separate obligations

The coarse index rebuilds its bucket map whenever a source is replaced or deleted. That makes mutation behavior straightforward to inspect, but it also means query timings say nothing about the cost of keeping the index current.

A production workload with frequent updates might care more about mutation cost or the visibility of new versions than about one isolated query. A mostly static catalog might make a different tradeoff. The current experiment does not choose between those workloads.

The generation guard addresses a related correctness question. After an index change, a caller naming the previous generation receives a refusal instead of querying an unacknowledged new snapshot. This makes the stale assumption visible even if the query vector itself is unchanged.

That is a local contract, not a general cache-coherence solution. The program does not coordinate replicas, retain multiple snapshots for readers or publish updates through a distributed index service. Those would be new mechanisms with additional evidence requirements.

The important lesson is that an index is not only a scoring function. It is also a maintained representation whose identity and update policy affect the meaning of every result. A search experiment that measures only steady queries can miss the conditions under which the correct document enters, leaves or changes inside that representation.

The approximate label is not a performance guarantee

An approximate method is often introduced to avoid scoring every vector. That motivation does not prove that every approximation is faster or worthwhile for every corpus.

The four-bit index can lose useful candidates because its partition is crude. Its update operation rebuilds buckets. Its query implementation allocates ordinary managed objects and sorts selected entries. None of those choices is optimized for a large deployment.

The benefit is explanatory clarity. If widening the radius restores the missing source, the experiment has isolated a candidate-selection problem. If a deleted source remains visible, the update contract is wrong. If exact and approximate methods both favor an inappropriate scope, the next question lies elsewhere.

The strongest counterargument is that a production HNSW implementation would behave differently. That is true, and the article does not claim otherwise. The same evaluation boundaries would still matter: exact-neighbor reference, task relevance, eligible scope, source freshness and the work included in timing.

The practical lesson is to resist solving every search problem by changing a similarity metric. First establish that the right representation exists, remains current, is eligible and reaches the candidate set. Only then can its score explain the final ordering.

Source code and execution evidence

Download the complete lab, source record and raw execution results. This package supports the specific execution claims in this article. It is not a production certification.