> ## Content Index > Fetch the complete content index at: https://devinside.cloud/llms.txt > Use this file to discover other available public pages before exploring further. # Technical Due Diligence for an Agentic Product - URL: https://devinside.cloud/technical-due-diligence-for-an-agentic-product/ - Published: 2025-11-25T12:45:00.000Z - Updated: 2026-11-12T17:08:26.000Z - Description: Diligence should connect differentiation to data access, quality, economics and failure recovery. - Author: Marcin Borecki - Tags: Advisory, Technical Advisory & Product Strategy, #m13, #devinside-review START WITH THE MECHANISM Interactive companion and laboratory update: 12 November 2026\. The editorial date places the topic on the timeline; this current explanation and its measured evidence retain their actual preparation and execution dates. THE FULL INVESTIGATION ## Technical Due Diligence for an Agentic Product Editorial context: 25 November 2025 · Prepared and updated: 12 November 2026 Dates and technology context This is a retrospective editorial timeline. Topic context: 25 November 2025\. First publicly available: 12 November 2026\. Current text, visuals and laboratory revisions were prepared on 12 November 2026. The reference due-diligence dossier examines later first-party laboratory artifacts and negative paths, not an investment review performed in November 2025. A product can be technically impressive and commercially undifferentiated. It can also have a crude interface, a modest model and a defensible place inside an important workflow. Technical due diligence becomes useful when it helps distinguish those possibilities without pretending that source code contains the answer to an investment decision. The phrase “agentic product” makes the exercise harder because it can conceal very different systems. One product may route requests through a fixed sequence of model calls. Another may let a model select tools, plan steps and act on external systems. A third may mainly package access to an underlying model with useful integrations. Each can create value. Each exposes a different set of claims to inspection. The proposed method here is to connect each material claim to an observable mechanism, a source of uncertainty and a decision that the evidence could change. It is a technical advisory framework, not an account of the author's transactions or a recommendation to invest. The examples are hypothetical. Their purpose is to show what an investigator should ask for and how to resist conclusions that the evidence cannot carry. ## Start with the decision the review is allowed to support Before opening a repository, establish the scope. Is the review intended to assess whether a pilot can reach production, whether the current architecture can support an agreed workload, or whether the product's claimed advantage depends on capabilities that are easy to replace? Those are related questions, but they are not interchangeable. A bounded review cannot certify every future use of a product. It can describe the examined version, the information provided, the tests performed and the assumptions under which its conclusions hold. State which materials were unavailable. If no production data can be shared, a review based on synthetic examples may still identify architectural risks, but it cannot establish real-world accuracy. Disclose relevant relationships and incentives. A reviewer who may later implement the recommended changes has a different position from one delivering an independent technical opinion. That relationship need not invalidate the work, but concealing it makes the report harder to interpret. The reader should know who supplied the evidence and who benefits from the proposed next step. The review also needs an audience. A founder may need a sequence of engineering decisions. An investment team may need to understand which claims remain unsupported. A buyer may need an operating boundary. Write for the decision each reader can actually make, rather than producing a document that sounds comprehensive because it contains every architecture term. ## Decompose the product's claim 01 / See the idea ### Turn a pitch into a testable claim ![A broad product claim is narrowed into a task, invariant and evidence request.](https://devinside.cloud/content/images/2026/11/devinside/M13-v1.svg) [Enlarge diagram](https://devinside.cloud/content/images/2026/11/devinside/M13-v1.svg) Suppose a procurement product claims to understand context. A review can turn that phrase into a policy-version case and an invariant that an unapproved supplier must never be treated as approved. Interpretation and evidence - Name one customer task. - Specify an observable failure. - Ask for evidence of that boundary. conceptual/hypothetical, not measured Consider a hypothetical agent that prepares procurement recommendations from internal documents and supplier information. Its pitch is that it understands context, reduces manual work and adapts to changing requirements. Those claims should be separated before they are tested. “Understands context” might mean retrieving the correct policy version, preserving a budget constraint or recognising that two business units use the same word differently. “Reduces manual work” might refer to searching, drafting or approval. “Adapts” might mean a configurable workflow, a model-selected sequence of tools or an engineer changing a prompt between demonstrations. Write each claim in a form that could fail. For example: given an approved set of procurement cases, the system produces a recommendation with the applicable policy citation and does not treat an unapproved supplier as approved. That statement still needs a test set and rubric, but it is far closer to something inspectable than “enterprise-grade intelligence.” Anthropic's distinction between predetermined workflows and model-directed agents helps identify which part of the control flow is actually dynamic. [Building effective agents](https://www.anthropic.com/research/building-effective-agents?ref=devinside.cloud). The diligence question is not whether the product deserves the fashionable label. It is which decisions the software delegates and whether the surrounding controls match their consequences. ## Inspect what the product knows and what it is allowed to know Data access can be a source of value, a dependency or a liability. Begin with the path from source material to an answer. Identify ingestion, transformation, indexing, retrieval and retention. Then ask who controls each stage and which changes the product can detect. The hypothetical procurement agent may work well because a knowledgeable employee curates a policy folder every week. That curation is a legitimate operating mechanism. It becomes a problem only when the product's economics or differentiation assume the work does not exist. The review should make the dependency visible and test what happens when it is delayed or performed incorrectly. Permissions need the same treatment. Does the retrieval path preserve the requesting user's access boundary? Can an administrator inspect material that ordinary users cannot? What enters logs, evaluation sets and support exports? A diagram should identify the trust boundary at each movement of data, rather than assuming that an authenticated front end settles every downstream decision. Do not make a legal conclusion from a technical inspection. Record which rights and approvals the product relies on, the evidence supplied for them and the person responsible for resolving uncertainty. A technical report can demonstrate that a source is ingested without proving that the organisation has every right required to use it. Keeping those propositions separate makes the report more useful to the people qualified to decide them. ## Test whether the evaluation can contradict the pitch An evaluation is valuable when a plausible failure would be detected and reported. Ask who wrote the cases, when they were selected and whether the developers used them while tuning the system. A test set repeatedly used to refine prompts is a development tool. It should not be presented as independent evidence of generalisation. Inspect a small number of cases in detail. Follow the expected result back to a source. Review an accepted answer, a rejected answer and a disagreement between reviewers. The aim is to understand the judgement procedure, not merely to reproduce a headline percentage. Pay attention to coverage. A product that handles a narrow class of requests reliably can be valuable if it identifies that class and routes everything else appropriately. A product that fails silently outside the demonstration's favourite cases creates a different operating burden. Evaluation should make the boundary of competence visible. The NIST generative AI profile offers a structured way to consider risks across the lifecycle, including evaluation and the consequences of system behaviour. [NIST AI 600-1](https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf?ref=devinside.cloud). It can inform the review's questions, but a framework reference is not evidence that the product satisfies them. The evidence must still come from the examined system. ## Substitute the model on paper, then in a bounded test 02 / A concrete example ### Portability is not a verdict ![A two-axis decision matrix separates replacement effort from the source of product value.](https://devinside.cloud/content/images/2026/11/devinside/M13-v2.svg) [Enlarge diagram](https://devinside.cloud/content/images/2026/11/devinside/M13-v2.svg) Imagine two products that can both swap models easily. One preserves a valuable workflow and the other exposes only a generic answer box; replacement effort alone cannot tell an advisor which product has useful differentiation. Interpretation and evidence - Inspect the remaining workflow. - Measure the adaptation work. - Keep the commercial conclusion separate. conceptual/hypothetical, not measured A useful thought experiment asks what remains if a competitor receives access to a model with similar capabilities tomorrow. The answer may be substantial: integrations, domain-specific evaluation data, trusted distribution, workflow fit or hard-earned operational knowledge. It may also be surprisingly small. Do not confuse the thought experiment with a claim that all models are equivalent. They differ in quality, latency, interfaces and failure behaviour. The point is to identify which part of the product's advantage belongs to the underlying provider and which belongs to the product team. A bounded substitution test can make that distinction more concrete. Hold the task set, retrieval configuration and acceptance rubric constant. Replace the model through an explicit adapter and record the engineering changes required. A supposedly provider-neutral architecture that requires extensive hidden prompt repair may be more coupled than its component diagram suggests. Conversely, easy substitution is not automatically evidence of weak differentiation. It can be a valuable architectural property if the product's advantage lies elsewhere. The report should say what the substitution reveals. It should not score dependency as bad and portability as good without considering the business and operating consequences. ## Follow a failure past the model call The most important failure may happen after the model has produced a perfectly reasonable plan. A tool can time out, a permission can expire or an action can succeed without the acknowledgement reaching the caller. The system then has to decide whether to retry, wait, compensate or ask a person. Ask the team to demonstrate one of those uncertain states in a controlled environment. The demonstration should identify the operation, the attempted effects, the durable record and the recovery route. A colourful trace is helpful, but it is not a substitute for establishing what actually changed in the destination system. Recovery ownership matters. If every unusual state requires the founder to inspect a database manually, the product may be operable at its present size while remaining dependent on a person who is not included in the cost model. That is a finding to investigate, not an excuse to invent a future scaling failure. Ask how frequently the intervention occurs and what would have to change for someone else to perform it safely. A human approval screen should also be examined as a system component. Does it show the proposed action, its target and the evidence behind it? Can an approval be reused after the action changes? Can a reviewer tell whether the action was merely proposed or actually completed? These questions reveal whether “human in the loop” is a real control or a reassuring phrase. ## Examine operating economics without manufacturing certainty The technical review can help identify the cost drivers that a commercial model depends on. It should not transform a small laboratory run into a confident forecast for a future market. For the hypothetical procurement agent, relevant drivers might include document refresh frequency, retrieval volume, model calls per task, retries, review effort and the proportion of requests requiring specialist escalation. Ask which of those have been observed and which remain assumptions. Keep measured values attached to their workload and execution date. Use scenarios to expose sensitivity. If the model price falls, does human review still dominate? If the corpus doubles, does ingestion work or retrieval quality become the limiting factor? If the supplier API changes, is the adaptation a configuration update or a substantial engineering project? The value of the exercise is identifying the assumption that most deserves a test. A severe limitation can sometimes be inexpensive to resolve. A modest-looking dependency can sometimes dominate the business. Ranking risks by how technical they sound will miss both. Connect each finding to a possible consequence, a proposed test and an owner who can decide whether that consequence matters. ## The counterargument: early products cannot provide mature evidence 03 / Watch the mechanism ### Missing evidence and adverse evidence ![A claim branches into an untested state or a contradicting observation with different next actions.](https://devinside.cloud/content/images/2026/11/devinside/M13-v3.svg) Missing evidence and adverse evidenceA claim branches into an untested state or a contradicting observation with different next actions. Animated explanation, not an execution recording.M13 / PROCESSMissing evidence and adverse evidenceRecovery claimNo run yetDuplicate effectTargeted experimentRepair or narrow scopeConceptual / hypothetical. Not measured. Play animation[Enlarge diagram](https://devinside.cloud/content/images/2026/11/devinside/M13-v3.svg) Static view. Play to see how the mechanism unfolds. Suppose recovery has never been exercised: that creates an unknown and a targeted test. If a controlled retry creates a duplicate effect, the result is adverse evidence and calls for repair or a narrower promise. Interpretation and evidence - An unknown is not a failed test. - A contradiction needs a response. - Stop once the agreed decision is supported. conceptual/hypothetical, not measured An early team may reasonably have little production history. Its evaluation set may be small, its observability incomplete and its operating process informal. Treating those conditions as proof of a poor product would make diligence a filter for documentation capacity rather than technical potential. The answer is to distinguish missing evidence from adverse evidence. Missing evidence means the review cannot yet support a claim. Adverse evidence means an observed result contradicts it. They require different responses. One may justify a targeted experiment; the other may require redesign or a narrower product promise. A useful early-stage report can therefore be short and directional. It might conclude that the retrieval boundary is coherent, recovery is insufficiently specified, and model substitution has not been tested. The next commitment could fund a bounded proof of those points. That is more actionable than either an unqualified endorsement or a long list of generic enterprise requirements. Proportionality should also protect the team from endless investigation. Once the agreed questions have enough evidence for the decision, stop. New concerns should be material and explicit. The purpose of diligence is to reduce consequential uncertainty, not to create an indefinitely expandable performance of caution. ## Write a technical opinion with edges The final opinion should distinguish what was demonstrated, what was inferred and what remains unknown. Attach the examined version, access limitations and actual test dates. Describe an important finding in a way that another engineer could reproduce or challenge. A useful conclusion might say that the examined product supports a bounded read-only workflow, while action-taking remains dependent on untested recovery behaviour. It might identify a defensible data preparation capability but insufficient evidence that the current evaluation measures it. It might find that a supposed model advantage disappears under a controlled comparison, while the integration and distribution advantages remain untouched. Those are technical contributions to a larger decision. They do not answer whether a valuation is attractive, whether a founder will execute well or whether a market will develop as hoped. They do make it harder for an attractive demonstration to carry claims it has not earned. The discipline is to keep every conclusion attached to its boundary. A strong review does not eliminate uncertainty. It helps the decision-maker see which uncertainty has changed, why it changed, and what evidence should be requested before the next promise becomes an obligation. ## A later reference dossier puts the review method to work *Contemporary laboratory note, prepared 12 November 2026\. The original November 2025 argument remains a review framework. The following dossier examines later first-party reference laboratories, not a customer, acquisition target or investment transaction.* The dossier combines evidence from several small implementations: local retrieval, a cost ledger, an action-reservation ledger, and selected runtime and telemetry laboratories. They are not one integrated product. Treating them as a complete tested agent would create exactly the kind of unsupported claim this review method is meant to prevent. The shared packet assigns stable identifiers to technical claims and preserves the relevant artifacts with SHA-256 hashes. A small Python script checks the snapshots, verifies selected predicates and emits an accepted, rejected or unknown status for each precisely worded claim. Those statuses are authored technical judgments whose stated predicates are checked by code. The script is not an automated diligence expert. It cannot decide whether a market exists, whether a founder will execute well or whether a passage fairly supports every possible interpretation. Its practical value is traceability. A reader can move from a status to the exact observation used, see the limitation and reproduce the simple check. The reviewer no longer needs to ask the audience to trust a summary detached from its inputs. This makes a useful first dossier because the evidence is narrow and visible. We can examine an actual result without pretending it represents the operating history of a mature company. ## Start with a claim that the evidence can contradict 01 / See the idea ### One adverse result narrows the promise ![One adverse result narrows the promise. The recorded top1 misses contradict the universal first-result claim. They do not establish whether a later answer would be correct, so that separate claim stays unknown.](https://devinside.cloud/content/images/2026/11/devinside/M13-packet-1.svg) [Enlarge diagram](https://devinside.cloud/content/images/2026/11/devinside/M13-packet-1.svg) The recorded top1 misses contradict the universal first-result claim. They do not establish whether a later answer would be correct, so that separate claim stays unknown. Interpretation and evidence - Reject the exact claim contradicted by evidence. - Preserve the narrower useful capability. - Do not turn missing evidence into an adverse result. Actual retrieval outcomes from the preserved packet. Claim C03 says that every first-ranked document was relevant in the declared retrieval fixture. The saved [retrieval evaluation](https://devinside.cloud/rag-quality-as-a-distribution-not-a-single-score/) contradicts that statement: the lexical method had three first-result misses and the dense method had two across the 18 evaluation queries. The claim is therefore rejected. That does not mean the retriever is useless. Both methods included all manually required evidence within their top three on this small set. The adverse observation narrows the supported promise: retrieving useful evidence somewhere in a short list is different from reliably placing the correct evidence first. The distinction matters for a product design. If a downstream component copies the first number it sees, the ranking error can become consequential. If the interface helps a person inspect several sources, the same retrieval result may still support useful work. The review should identify that dependency rather than jump from one failed claim to a verdict on the entire idea. The fixture also has a limitation that must remain visible. Its questions and relevance judgments were manually authored within the same project. They were fixed before inference and were not tuned afterward, but they are not an independent sample of production users. A defensible opinion can accept the observed rank result, reject an overbroad first-result claim and still leave product usefulness unknown. Those positions are compatible because they refer to different propositions. ## Follow authority through the receiver, not only the interface 02 / See the idea ### A lost reply does not release the obligation ![A lost reply does not release the obligation. The saved authority trace retains the reservation while the receiver outcome is uncertain. Reconciliation changes held units to committed units while the receiver-effect count stays one.](https://devinside.cloud/content/images/2026/11/devinside/M13-packet-2.svg) [Enlarge diagram](https://devinside.cloud/content/images/2026/11/devinside/M13-packet-2.svg) The saved authority trace retains the reservation while the receiver outcome is uncertain. Reconciliation changes held units to committed units while the receiver-effect count stays one. Interpretation and evidence - Inspect the receiver as well as the caller. - Preserve uncertainty until evidence resolves it. - Do not infer power-loss coverage from database reopen. Executed AI13 synthetic authority trace; no real payment. The [action-reservation laboratory](https://devinside.cloud/agent-spending-authority-reservations-approval-scope-and-uncertain-charges/) provides a concrete chain to inspect. Its saved example begins with 100 synthetic accounting units available, holds 70 for an operation, loses the response after the receiver has recorded one effect, and later reconciles after reopening the database. The packet's C07 accepts a narrow claim about that recorded scenario: the reservation remains held during uncertainty and the receiver-effect count remains one after reconciliation. The values are fixture units, not money moved through a real payment service. This supports a useful distinction. A lost reply is not evidence that the receiver did nothing. Releasing the reservation merely because the caller lacks confirmation would erase a possible obligation. The recorded state instead preserves uncertainty until evidence from the receiver resolves it. The review must stop at the boundary of the experiment. The checks use a local SQLite fixture and a separate receiver database. They do not establish bank integration, power-loss behavior across every storage condition, administrator tamper resistance or a complete production identity system. Claim C08, which asks whether recovery has been demonstrated for arbitrary power loss and all operational failures, therefore remains unknown. It is not accepted because some recovery tests passed, and it is not rejected because those broader tests are absent. This is how a technical review becomes useful without pretending to certify everything. It names the demonstrated transition and the next unresolved operating condition separately. ## A negative result can improve the product claim The packet also examines the [reusable asynchronous-source laboratory](https://devinside.cloud/behind-valuetask-a-custom-ivaluetasksource-and-its-reuse-protocol/). A proposed claim says that reducing allocation made the reusable source faster in every measured mode. The saved ready-mode measurements contradict it: the source's median elapsed time exceeds the task baseline's median in that fixture. The broad speed claim is rejected while the narrower allocation observation remains available. This is not a failure of the evidence pack. It is a successful correction of the story the evidence can support. The same habit is relevant to an agentic product. A system can reduce model calls while increasing review effort. It can improve common retrieval cases while placing a more consequential exception incorrectly. It can shorten one local operation without changing end-to-end completion. A useful review asks which promise should change after such a result. Perhaps the implementation belongs only on a particular hot path. Perhaps the interface should expose more than one source. Perhaps the product should begin as drafting assistance rather than autonomous action. The technical opinion should not conceal these narrower opportunities behind an all-or-nothing verdict. A rejected universal statement can leave a valuable bounded capability intact. The point is to describe that capability precisely enough that the next commitment is made with fewer hidden assumptions. ## Do not combine separate successes into an untested system The local retrieval experiment, synthetic cost ledger and action-reservation ledger each have useful evidence. Their individual results do not establish that an integrated agent preserves all three contracts at once. Integration creates new questions. Does a retrieved document's tenant scope remain connected to the actor who requests an action? Does the task identifier used for cost accounting survive retries and reconciliation? Can a model-generated proposal change between the evidence bundle and the approval record? Does cancellation leave the cost ledger and action ledger describing compatible states? These are not accusations that the components fail. They are claims for which the dossier lacks integrated evidence. C13 records that gap explicitly. The next technical commitment could be a small composition test with one synthetic workflow and a controlled receiver. It should carry identifiers and authority through the interfaces, then introduce a failure that crosses more than one component. The existing artifacts would help define its acceptance conditions, but would not replace execution of the composed system. This boundary protects both reviewer and builder. The reviewer avoids selling a collection of component successes as a product assurance. The builder receives a focused next experiment rather than a vague demand to become enterprise-ready. ## Check the evidence's identity before interpreting its meaning The packet includes a practical provenance lesson from the [measurement-contract article](https://devinside.cloud/same-service-different-numbers-a-measurement-contract-for-net/). Its accepted evidence identifies the actual benchmark runtime and retains an earlier attempt rejected for runtime drift. A configured version and an executed version are not automatically the same fact. The [telemetry laboratory](https://devinside.cloud/the-observer-effect-a-small-net-telemetry-lab/) supplies another boundary. Its evidence records transport acknowledgement. That does not automatically establish durable downstream storage, queryability or retention under every failure condition. A reviewer should inspect those distinctions before using the artifacts to support a wider promise. The same applies to [runtime-async](https://devinside.cloud/runtime-async-in-net-11-who-builds-the-state-machine-now/) and [escape-analysis](https://devinside.cloud/the-allocation-that-disappears-escape-analysis-from-net-10-to-net-11/) results: compiled representation, local allocation and application performance are different observations. These examples are later technical evidence, not facts available to the original 2025 review context. They illustrate why a dossier should preserve the exact artifact, actual execution conditions and the claim's scope. A hash identifies which bytes were reviewed; the interpretation still needs an accountable person. ## End with a bounded next decision The completed packet contains 13 claims: six accepted within their stated boundaries, two rejected by specific observations and five unknown. These counts are an inventory, not a quality score. Six accepted claims do not compensate for one unknown that is essential to the next action. For example, local retrieval and controlled reservation recovery might justify building an integrated reference workflow. They do not justify claiming customer savings or autonomous handling of real financial effects. The missing evidence changes the permissible next commitment. This also answers the concern that diligence can become endless. The next step should target the uncertainty that matters now. A team exploring a read-only assistant may not need a production payment integration. A team proposing consequential autonomous action cannot treat that same gap as irrelevant. The dossier's value is the clarity of that distinction. It provides code, artifacts and explicit limits without pretending to decide the investment. A serious technical opinion can be concrete, useful and incomplete at the same time, provided it tells the decision-maker exactly where the incompleteness begins. [Download the complete laboratory and execution records](https://devinside.cloud/content/files/2026/11/M13-lab.zip).