An Evidence-Aware Agent Framework Upgrade Tracker in .NET

START WITH THE MECHANISM

Interactive companion and laboratory update: 12 November 2026. The editorial date places the topic on the timeline; this current explanation and its measured evidence retain their actual preparation and execution dates.

THE FULL INVESTIGATION

An Evidence-Aware Agent Framework Upgrade Tracker in .NET

Dates and technology context

This is a retrospective editorial timeline. Topic context: 12 November 2026. First publicly available: 12 November 2026. Current text, visuals and laboratory revisions were prepared on 12 November 2026.

Keep in the current release window.11.0.0 published10Nov2026; actual selected-contract comparison ran12Nov2026. Keep publication/observation/execution timestamps separate.

An executed comparison of selected Microsoft.Extensions.AI packages and scripted function scenarios. It is not a certification of every SDK feature or a live-model evaluation.

A package feed says that a new release exists. A dependency manager can restore it. A small example compiles. None of those observations, separately or together, proves that an application still behaves the way its users require.

The missing object is a relationship between a release and an application contract. Which behavior matters? Which exact artifact was inspected? Was the relevant scenario executed, or did someone only read a release note that sounded reassuring? When the evidence is incomplete, can the tracker say unknown instead of converting a version number into a compatibility badge?

This article implements a small C# tracker with that discipline. It reads public NuGet metadata for two explicitly selected Microsoft.Extensions.AI versions, records the source URLs and hashes, and combines those observations with separately compiled behavior runs. The versions in the fixture are 9.10.2 and 11.0.0. Both were restored and exercised on the same pinned .NET 10.0.15 runtime using SDK 10.0.400.

The behavior fixture uses actual IChatClient and automatic function invocation. Its provider is a local script that requests a controlled function. There are no remote model calls, tenant credentials, billing queries or account-limit checks. The result is deliberately narrow: the two packages produced equivalent outcomes for three declared scenarios in this environment.

That is useful evidence. It is also much less than a statement that every application can upgrade safely. The tracker keeps those two claims separate in its machine-readable report.

Define the decision before collecting versions

An upgrade tracker can become a list of newer numbers with an attractive interface. That is useful for discovery, but weak for deciding whether to change a working application. The important question is what the application relies on and what could change the next release decision.

For this reference, the application boundary is a model-requested function that enters a deterministic effect guard. An authorized actor should produce one synthetic effect. An unauthorized actor should produce none. A repeated function request with the same operation identity should not create another effect.

These scenarios are not a complete agent system. They do not cover streaming, sessions, multimodal content, prompt behavior, provider retries or every serializer option. They identify a small contract that matters when changing the framework plumbing around a controlled application action.

The tracker therefore has two sources of evidence. Public release observations establish package identity and publication metadata. Local verification runs establish behavior under the declared fixture. A release note can inform which scenarios to add, but it cannot substitute for their execution.

This separation changes the output. Instead of one green compatible label, the report can say that the selected scenarios agree, that both projects compiled, that the exact packages were observed, and that general compatibility remains unknown. A decision-maker can then judge whether the tested boundary is sufficient for the intended change.

The same structure can scale to a larger application. Add scenario identities for cancellation, state migration, tool schema, authorization or persistence only when the application depends on them. The tracker should make those obligations visible rather than pretend that a package's semantic version automatically supplies them.

Keep publication, observation and execution dates distinct

01 / See the idea

Three clocks, three meanings

Enlarge diagram

The NuGet catalog gives a publication timestamp for a package version. The tracker records a separate checkedAt timestamp when it reads that source. The behavior report records when its executable scenarios ran. These dates answer different questions.

The observed publication times were 2 November 2025 at 17:55:38.61 UTC for 9.10.2 and 10 November 2026 at 23:48:23.46 UTC for 11.0.0. The tracker checked both on 12 November 2026. Those observations do not imply that the older package was first examined when it was published or that the behavior test ran at either release date.

The timestamp comes from the package catalog's published field. The code does not substitute an HTTP modification date, repository commit date or documentation crawl date. A later change to the package page's presentation should not silently move the release's historical position.

The version label is also kept modest. A version without a prerelease suffix is classified as a stable-version label. That describes the package naming, not a guarantee of reliability, support duration or compatibility with every consumer. A stable release can still change a behavior the application needs.

Official NuGet resources distinguish registration information from the package archive itself. The tracker follows those specific resource relationships for the two chosen versions instead of crawling arbitrary links from a release description. NuGet registration resource, package content resource.

An honest date model also helps during maintenance. If the tracker is rerun after a scenario changes, the new execution date should be attached to the new source identity. Editing a report's explanation should not imply that the package was reinstalled or the application retested.

Limit the metadata reader to known public sources

The CLI constructs registration and package-content URLs for Microsoft.Extensions.AI and the two declared versions. It accepts HTTPS resources only on api.nuget.org. The catalog link returned by registration is checked against that same host restriction before fetching.

This is intentionally not a general release crawler. An arbitrary URL in release notes must not become an instruction to fetch private addresses, follow unknown hosts or execute a setup command. Release text is data to preserve and interpret, not a source of authority over the tracker.

The HTTP reader has a timeout and bounded response sizes. Registration and catalog responses are limited to four mebibytes; the package archive has a sixteen-mebibyte fixture limit. Those are local safety bounds for this selected input, not provider account quotas or claims about the largest legitimate NuGet package.

The code reads responses as streams and stops if the selected bound would be exceeded. It does not download an unlimited body and check its length afterward. A failed read is recorded as unavailable for the handled transport and size failures, leaving behavior inference unknown rather than silently omitting the version.

The allowlist is not a complete network-security product. It is a narrow boundary for a tool whose immediate purpose is known. The code does not implement arbitrary redirects, package discovery across registries or authentication to private feeds. Those would require additional source and credential policies.

The actual run successfully read both registration records, both catalog records and both package archives. The resulting report contains their source URLs, publication times, observation times and SHA-256 values. No secret-bearing endpoint was involved.

Hash the representation you actually inspected

02 / See the idea

Identity is not compatibility

Enlarge diagram

A URL names a resource, while a hash identifies the bytes read from it. The tracker records separate hashes for registration JSON, catalog JSON and the package archive. These are different representations with different reasons to preserve them.

The registration hash helps identify the exact link and metadata response used during observation. The catalog hash identifies the release record whose publication timestamp and description were interpreted. The archive hash identifies the downloaded package bytes, rather than assuming that the version string alone is a complete record of what was inspected.

The relevant operation is ordinary cryptographic hashing:

string packageSha256 = Convert.ToHexString(
    SHA256.HashData(payload));

That operation does not verify a package signature or establish publisher trust by itself. It supplies a representation identity for the evidence record. Authenticity and repository trust are separate concerns. A tracker should not call a matching hash a security attestation unless it has a trusted expected value and a defined verification process.

The package restoration used by the behavior projects is recorded through each project's lockfile. The downloaded archive observation and the restored dependency graph are related artifacts, but the report does not pretend that hashing the top-level archive checks every transitive package. A real upgrade review should inspect the resolved graph relevant to the application.

The behavior source is hashed as well. Both projects contain identical C# source, and the final acceptance check confirms byte-for-byte equality. This reduces the chance that the old version receives an easier test or the new version receives a subtly modified expectation.

These identities give a reviewer a useful chain: selected package, observed metadata, restored graph, exact scenario source and executed result. That chain is much more informative than a screenshot showing that two builds happened to be green.

Compile the same contract against two real packages

05 / Watch the mechanism

From release bytes to a narrow decision

Enlarge diagram

Static view. Play to see how the mechanism unfolds.

The fixture has separate old and new projects. One references 9.10.2, the other 11.0.0. Both target net10.0 and use the same SDK and runtime pins. They do not share a single output directory whose last restoration could accidentally determine both executions.

The comparison therefore changes the framework package boundary while holding the scenario source and runtime constant. It is not a runtime migration benchmark, and it is not evidence that the older package supports every earlier runtime the organization might use.

The application registers a small function using the actual API:

string Execute(string operationId)
{
    if (trustedActor != "alice")
    {
        trace.Add("denied");
        return "denied";
    }
    if (!seen.Add(operationId))
    {
        trace.Add("replayed");
        return "replayed";
    }
    effects++;
    trace.Add("applied");
    return "applied";
}

The function is exposed with AIFunctionFactory, and FunctionInvokingChatClient performs the tool call requested by the scripted provider. This is a real integration path. The test would catch a binding or invocation difference that a direct call to Execute could miss.

The provider does not choose a useful action through reasoning. It emits the exact function request needed by the scenario, including an identity-looking extra argument. Authority still comes from the host-side captured actor, so a model-supplied string cannot make the denied scenario pass.

The Microsoft abstraction defines the chat and tool-call surface; the application retains the effect policy. That separation is the contract the upgrade test is intended to preserve. Microsoft IChatClient guidance.

Preserve the difference between an attempt and an effect

The authorized scenario asks for one operation and expects one effect. The denied scenario supplies a host context for another actor and expects no effect. The replay scenario asks for the same operation twice and expects one effect plus a replay observation.

The scripted provider performs two rounds for ordinary authorized and denied cases: one function request and one final response after the tool result. The replay scenario performs three rounds because it requests the same operation again before returning its final message.

Those counts are part of the fixture's observed behavior. They are not token costs, latency estimates or evidence that a real model will use exactly that many calls. They help detect a changed invocation path in this controlled comparison.

The expected effect count is independent of the SDK's final text. A framework could return a friendly completion sentence after an unauthorized action; the recorder would still expose the effect. Conversely, a different harmless final phrase should not make the domain comparison fail if the contract deliberately ignores presentation.

The ordered application trace is retained alongside the count. Authorized execution records applied. Denial records denied. Replay records applied followed by replayed. Comparing those traces helps locate a change rather than reducing the result to one pass flag.

Each project passed six assertions across the three scenarios. The report confirms that the scenario objects are equal between the old and new runs. This is a bounded behavior result, not a claim that every internal event, response field or performance characteristic is identical.

Let release notes remain insufficient evidence

The tracker preserves the catalog's release-notes field if present. It does not attempt to convert a sentence such as improved reliability into an automatic pass for the application's recovery contract.

Release notes can identify a likely change, an intended fix or a feature worth investigating. They may not describe every affected overload, transitive dependency or edge case. Even a precise note about one behavior does not establish that the application's composition of components still satisfies its own requirements.

The report therefore marks behavior inferred from release notes as unknown. It separately records the executed scenario comparison. This avoids the common mistake of presenting documentation evidence and runtime evidence as equivalent green checks.

A missing note is also not proof that behavior did not change. It may mean that the release record is sparse, that details are elsewhere or that an implementation change was not described at the level the application needs. The appropriate next action is a targeted investigation, not a guessed compatibility conclusion.

This conservative rule does not make the tracker passive. The report can still say which known contract has been executed and which remains unverified. It becomes a planning artifact for the next regression test rather than an oracle pretending to know every consequence of an upgrade.

Read the actual comparison without expanding its claim

03 / A concrete example

The same scenarios, two packages

Enlarge diagram

The accepted local result is straightforward:

Scenario	Package 9.10.2	Package 11.0.0	Required effect
Authorized operation	Applied, two provider rounds	Applied, two provider rounds	One effect
Unauthorized actor	Denied, two provider rounds	Denied, two provider rounds	Zero effects
Replayed operation	Applied then replayed, three rounds	Applied then replayed, three rounds	One effect total

Both projects compiled and ran on .NET 10.0.15. Their scenario source is identical. The public metadata reader successfully observed both selected releases and stored the hashes of the inspected representations. The comparison field is true for the declared scenarios, while generalCompatibility remains unknown and liveModelQuality remains not tested.

There is no performance result in this experiment. A metadata-fetch duration would mostly describe network and cache conditions, while a scripted chat duration would not represent a hosted model. Neither would answer the upgrade contract being tested, so no such number is promoted into the report.

The result also does not cover serialization of persisted sessions, streaming interruption, concurrent use, cancellation or provider-specific connector behavior. Those are material omissions for an application that depends on them. The tracker should add their scenario identities before claiming readiness for that application's upgrade.

Treat unknown as a useful state in the interface

04 / See the idea

The decision keeps an unknown branch

Enlarge diagram

A reader-facing view can be built directly from the JSON without pretending that every field is equally authoritative. Show the package publication date, the observation timestamp, the execution timestamp and the scenario status in separate columns. Link the source URL and the relevant run artifact beside the claim they support.

A stable-version label should not share the same visual treatment as an executed acceptance test. Likewise, a source fetch failure should not disappear from the release list. It should remain visible with its error category and an unknown status for claims that depend on it.

The fixture does not implement a full dashboard. Its JSON is the machine-readable interface and its article figures show the intended distinctions. A production UI would need accessibility, stale-data presentation and filtering appropriate to its users, but those additions should preserve the evidence model rather than collapse it into a single upgrade score.

This is especially valuable when several teams use the same framework differently. One team's tool-binding scenario can pass while another team's streaming contract remains untested. The release itself is neither globally approved nor globally rejected by that partial evidence. Each consumer needs the subset of contracts it actually owns.

The tracker can therefore become a shared index of decisions without becoming a central authority that invents compatibility. It records who checked what, against which artifacts, and what the result permits the team to conclude.

Challenge the scope before accepting the result

Consider a deliberately uncomfortable example. Suppose an application uses the framework only to classify a document. Its important invariant is that an unavailable classifier cannot produce an accepted classification. The three tool scenarios in this repository say almost nothing about that application. Reusing our green result would be a category error even though the package name matches. The correct next step is to create a classification contract, preserve its own expected outcomes, and run it against both candidates. Evidence is portable as a method; its conclusion is portable only when its boundary matches.

Now suppose another application uses tools to reserve stock. Our replay scenario is relevant, but it is still insufficient. The fixture suppresses a second operation within one process. A real reservation may need to survive a process restart and an ambiguous network response. Those requirements change the location of the invariant. The operation identity must reach the durable reservation boundary, not merely a chat wrapper. The tracker can point to that test, but collecting package metadata does not implement it. Separating these responsibilities prevents a release report from absorbing authority it has not earned.

There is a serious counterargument to this approach: maintaining contracts costs time, and a small team may reasonably prefer ordinary dependency updates with existing regression tests. That is often a good choice. This tracker is not a reason to build a second test system beside a working one. Its useful role is to connect existing tests to the specific decision being made and preserve why a particular version was accepted. If the current pipeline already records package locks, source revisions, runtime details and consumer behavior, the tracker should consume those artifacts rather than duplicate their execution.

The extra structure becomes worthwhile when release information and application evidence are routinely confused. A team that says the package is stable, therefore the workflow is safe, needs a clearer boundary. A team that says one payment tool passed, therefore every tool is compatible, needs a narrower scenario identity. A team that refuses an upgrade because nobody can prove every possible behavior needs a practical account of what the application actually depends on. In each case, the benefit is better reasoning rather than more green icons.

The report also makes rollback less ambiguous. Imagine that the new candidate passes these three scenarios but later fails a streaming scenario added by a consumer. The release record should not erase the earlier successes or relabel them as failures. It should retain the passing contracts, add the failing contract, and explain why that consumer cannot adopt the candidate. Returning to the previous package version is a deployment decision with its own state compatibility concerns. Our fixture has no persisted sessions and therefore offers no evidence that downgrading a real session store is safe.

One useful extension would be a machine-readable contract manifest with identifiers such as tool.authorized.single-effect and tool.replay.same-operation. Each identifier would declare its source revision, input fixture and expected observable result. A comparer could then distinguish a missing scenario from a failed scenario and a renamed scenario from a genuinely new requirement. This is a proposed extension, not implemented functionality. The current comparison deliberately remains small enough that a reader can inspect the complete scenario arrays.

Another extension would preserve failed executions as first-class artifacts. The current demonstration stores successful runs and aborts on a violated assertion. A continuous application pipeline should also preserve compilation failure, restore failure, test failure and unavailable evidence separately. A compile error might establish an API incompatibility in the selected source. A network failure during package retrieval establishes no such incompatibility. Collapsing both into a single red state loses precisely the information needed to choose the next action.

Reviewers should also challenge the independence of the oracle. In this fixture the expected effect counts are written explicitly rather than obtained by running the old version and treating whatever it does as correct. That distinction matters. If both package versions preserve an application bug, equality alone produces a comforting but useless result. The authorized case expects one effect, the denied case expects zero, and the repeated operation expects one. Equality is then a comparison between independently checked results rather than an accidental agreement between two executions.

Even those assertions are not a proof of authorization for arbitrary identities. The scripted actor fixture contains a small trusted-context boundary and deliberately hostile extra arguments. It demonstrates that this particular function binding does not let those arguments replace the captured actor. A real application must establish the actor before that boundary and must decide which operation that actor may perform. Authentication, tenant mapping and resource authorization are outside this small program. Keeping them outside the claim is part of making the result useful.

Finally, freshness deserves its own decision. Re-running the metadata reader tomorrow may produce a new observation timestamp without changing the package archive. Re-running the behavior project after a compiler or operating-system update may produce a new execution with the same package versions. Neither should silently replace the previous artifact. A consumer can retain both and decide whether the changed environment matters. The current report is one bounded observation prepared on 12 November 2026, not a continuously maintained assurance service.

The trust boundary of the evidence file deserves similar care. Hashing a package archive gives a repeatable identity for bytes that were observed. It does not authenticate every claim written next to that hash, and it does not prove that a different machine executed those bytes. A mature pipeline could bind artifacts to a build identity and preserve signed attestations. This laboratory does not implement that machinery. Its evidence is a local, inspectable record whose source, commands and outputs can be reproduced.

The metadata reader also targets a particular public registry rather than implementing every possible NuGet server. The API documentation describes discovering resources from a service index; this bounded fixture uses the known nuget.org resource locations for its two declared candidates. Generalizing it to other feeds would require resource discovery, source policy and failure handling appropriate to those feeds. That extension should not be smuggled into the current claim merely because the returned JSON follows a familiar schema.

Reproduce the comparison before changing production

The two behavior projects are intentionally separate. Run each with its own package reference and lockfile, retain its JSON result, then run the tracker against those results and the shared source. The commands do not modify a production application or call a live AI endpoint.

dotnet run -c Release --project old/Behavior.csproj -- ../old-behavior.json
dotnet run -c Release --project new/Behavior.csproj -- ../new-behavior.json
dotnet run -c Release --project Tracker.csproj -- ../release-report.json `
    ../old-behavior.json ../new-behavior.json old/Program.cs

The tracker reads only its declared public registry resources. The behavior programs use a local script as the provider and dispose their client wrappers. There is no background service or scheduled updater left running when the commands finish.

Before applying this pattern to a real upgrade, choose the contract that would make the change unsafe or useless if it regressed. A test of one authorized tool call is not enough for an application whose core value depends on durable sessions. A test of session deserialization is not enough for a system whose risky boundary is an external effect.

The valuable artifact is the connection between a release and a decision. It says: these exact sources were observed; this exact code was compiled; these scenarios ran; this narrower behavior was preserved; everything beyond that remains a question. That is a stronger basis for an upgrade than either trusting a version number or refusing every change until impossible certainty arrives.

Source code and execution evidence

Download the complete lab, source record and raw execution results. This package supports the specific execution claims in this article. It is not a production certification.