Human Approval After a Restart: One Reviewed Proposal, One Local Effect

A reviewer opens the proposal for case T-1042: the selected source versions, the accepted findings, the exact reply and the policy that governed the work. The reply explains the known firmware issue and asks for a diagnostic export without recommending a rollback. The reviewer approves. The process writes the support review record, then crashes before it can return a receipt. The screen shows an error.

What should happen when the reviewer, or a retry loop, approves again? If the answer is "it depends on timing", the system will eventually create two records for one decision. This last part of the series makes the answer deterministic: one reviewed proposal, one operation identity, one local effect, recovered across a process restart with Microsoft Agent Framework checkpoints.

Three identities answer three different questions

Recovery designs go wrong when one identifier is asked to do three jobs. In this application there are three, and each is insufficient on its own:

A checkpoint can be replayed. A proposal can be reconstructed. Neither fact authorizes a second effect. Conversely, an effect can exist while the caller has no receipt and the workflow never recorded a successful return from its commit step. The lost receipt in the opening is exactly that state.

The effect in this application is deliberately local: a review record named record_support_review. No email, refund, firmware change or external ticket update happens. Keeping the effect local is what lets the recovery argument be complete; the last section explains what changes when the effect lives in someone else's system.

Specify canonical bytes, not "the same JSON"

The operation identity binds, in this order: a protocol tag decisionflow.operation.v1, the case ID, the action name, the canonical draft body, the selected-evidence identity from Part 8 and the effective policy identity from Part 9. How those fields become bytes is part of the contract.

Hashing display JSON makes the identity depend on indentation and property order. Concatenating strings without framing is worse: the fields ab, c and a, bc concatenate to the same text. The encoding here normalizes CRLF and lone CR to LF, leaves every other character alone, encodes each field as UTF-8 and prefixes it with its byte length as a four-byte big-endian integer. SHA-256 over that byte sequence, with the prefix op-v1-, is the identity.

public static string Text(string s) => s.Replace("\r\n", "\n").Replace('\r', '\n');

public static byte[] Encode(IEnumerable<string> fields)
{
    using var stream = new MemoryStream();
    Span<byte> length = stackalloc byte[4];
    foreach (string field in fields)
    {
        byte[] bytes = Encoding.UTF8.GetBytes(Text(field));
        BinaryPrimitives.WriteInt32BigEndian(length, bytes.Length);
        stream.Write(length);
        stream.Write(bytes);
    }
    return stream.ToArray();
}

public static string Id(string caseId, string action, string body, string evidenceId, string policyId) =>
    "op-v1-" + Convert.ToHexString(SHA256.HashData(Encode(
        ["decisionflow.operation.v1", caseId, action, body, evidenceId, policyId]))).ToLowerInvariant();

Lengths count bytes, not .NET char values or JavaScript string units, which is why the test vectors include Polish text. Encoded by the C# code above and, independently, by a short Python script using str.encode('utf-8'), struct.pack('>I', ...) and hashlib.sha256, the vectors agree byte for byte:

ab|c                                  0000000261620000000163
a|bc                                  0000000161000000026263
Proszę zebrać eksport diagnostyczny.  00000026 50726f737ac499 ... 6e792e   (36 chars, 38 bytes)
line one\r\nline two                  000000116c696e65206f6e650a6c696e652074776f
line one\nline two                    000000116c696e65206f6e650a6c696e652074776f

The first two rows show why framing matters: same concatenation, different bytes. The Polish row is 36 characters and 38 bytes, because "ę" and "ć" take two bytes each. The last two rows show the one normalization that is applied. No trimming, case folding or Unicode normalization happens, so adding a sentence, changing a space or changing a source version changes the operation. A line-ending-only change does not, and the reviewer sees the canonical LF body that is stored.

A digest here is an identifier and a consistency check. It is not a signature, an authorization grant or a substitute for protecting the state directory. Someone who can replace the program and all of its records is outside what this design defends against.

A human edit is a new proposal

Editing is where approval systems quietly lose their meaning. If the reviewer can change the reply and then approve, which text did they approve, and which checks still apply? The rules that keep this honest:

• Saving an edit carries the expectedOperationId of the proposal on screen. If the current proposal has a different identity, the save is refused as stale.
• An accepted edit creates a new proposal with a new identity, explicitly marked as human-edited. The licensed-clause checks of Part 8 cover only the clauses they checked; they do not transfer to sentences a person wrote.
• Changing only a source version also creates a new proposal. That is the hard case worth testing: the visible text is identical, and an approval from an older browser tab must still be refused.
• A decision, once recorded, is immutable. Repeating the same decision is a replay. Changing approve to decline under the same identity is a conflict, not an update.

app.MapPost("/cases/{id}/approve", async (string id, Store store) =>
{
    var proposal = await store.CurrentProposalAsync(id);   // whatever is newest
    return await store.CommitAsync(proposal);               // no binding, no key
});

app.MapPost("/cases/{id}/approve", async (string id, ApproveRequest r, Store store) =>
{
    if (r.Approved is not bool approved) return Results.BadRequest();   // no implicit consent
    var current = await store.CurrentProposalAsync(id);
    if (current.OperationId != r.OperationId) return Results.Conflict("stale proposal");
    return Results.Ok(store.Commit(new ReviewDecision(r.OperationId, approved, r.Reviewer)));
});

Reviewer identity matters too, but it is a separate concern. In production the reviewer comes from the application's authentication and approval policy, attached at this endpoint, while the operation binding below stays the same. Giving that reviewer a decision they can still change, rather than a rubber stamp, is the subject of the human in the loop needs a job description.

The workflow, its pending request and the checkpoint

The workflow has three executors with fixed IDs: prepare builds the immutable proposal and its identity, human is a typed request port, and commit owns the local effect. In Microsoft Agent Framework, RequestPort.Create with the request type ReviewProposal, the response type ReviewDecision and the ID human-review-v1 declares a typed external request: when a message reaches the port, the run pauses with status PendingRequests and emits a RequestInfoEvent. The framework provides the request and response mechanism; what approval means, and whether a response is valid, is the application's job.

With a CheckpointManager, the framework writes a checkpoint at the end of every superstep, capturing executor state, pending messages and pending requests. The begin command runs until the review request, records the session ID, checkpoint ID and operation identity in a small pending.json, disposes the run and the FileSystemJsonCheckpointStore, and exits. A later process resumes from that checkpoint:

using var cpStore = new FileSystemJsonCheckpointStore(new DirectoryInfo(Path.Combine(dir, "checkpoints")));
var manager = CheckpointManager.CreateJson(cpStore);
await using Run run = await InProcessExecution.ResumeAsync(Build(store, lose),
    new CheckpointInfo(pending.SessionId, pending.CheckpointId), manager);

WorkflowEvent[] events = run.NewEvents.ToArray();          // read once
var req = events.OfType<RequestInfoEvent>().SingleOrDefault();
req.Request.TryGetDataAs(out ReviewProposal? p);
if (p!.OperationId != pending.OperationId) return 4;        // not the proposal we recorded

await run.ResumeAsync([req.Request.CreateResponse(
    new ReviewDecision(p.OperationId, approved, reviewer))]);
WorkflowEvent[] after = run.NewEvents.ToArray();
var failed = after.OfType<ExecutorFailedEvent>().FirstOrDefault();   // never treat silence as success

Three framework details showed up when running this, and each one shapes the design:

• The pending request is re-emitted on resume. After ResumeAsync the run is again in PendingRequests with the same proposal. That is convenient, and it means the same checkpoint can be resumed and answered any number of times. Nothing in the checkpoint prevents a second commit.
• Run.NewEvents moves a bookmark. It returns events since the last access. In a one-executor test the first read returned 6 events and an immediate second read returned 0. Read it once into an array, then look for the request, the output and failures in that array.
• The rebuilt workflow must match the checkpoint. After changing the commit executor's output type, resuming an older checkpoint failed with InvalidDataException: "The specified checkpoint is not compatible with the workflow associated with this runner." The documentation states the rule: a rehydrated workflow must keep the topology and executor identities, and agents inside it need a stable ChatClientAgentOptions.Id. Record a workflow version with every pending request and refuse to resume across an incompatible deploy.

The file store's own documentation describes it as durable, process-exclusive persistence whose instances are not thread-safe. That is a single-host design: take a local writer lock around begin, resume and reconcile, and do not point several production instances at one directory. Treat the checkpoint directory as trusted storage, because the documentation is explicit that checkpoints must never be loaded from untrusted or tampered sources.

Put the effect behind its own immutable key

The checkpoint cannot answer "did the effect happen?". The effect store must, so the effect is keyed by the operation identity and written once.

static Workflow Build(EffectStore store, bool loseReceipt)
{
    var prepare = new Func<CaseInput, ReviewProposal>(c =>
        new ReviewProposal(Operation.Id(c.CaseId, "record_support_review", c.Body, EvA, P(9)), c.CaseId, c.Body))
        .BindAsExecutor("prepare");
    var human = RequestPort.Create<ReviewProposal, ReviewDecision>("human-review-v1");
    var commit = new Func<ReviewDecision, CommitResult>(d => store.Commit(d, loseReceipt))
        .BindAsExecutor("commit");
    return new WorkflowBuilder(prepare)
        .AddEdge(prepare, human)
        .AddEdge(human, commit)
        .WithOutputFrom(commit)
        .Build();
}

bool WriteOnce(string path, object value)
{
    string tmp = path + "." + Guid.NewGuid().ToString("N") + ".tmp";
    try
    {
        using (var fs = new FileStream(tmp, FileMode.CreateNew, FileAccess.Write))
        {
            JsonSerializer.Serialize(fs, value);
            fs.Flush(flushToDisk: true);
        }
        File.Move(tmp, path, overwrite: false);
        return true;
    }
    catch (IOException) when (File.Exists(path)) { return false; }
    finally { if (File.Exists(tmp)) File.Delete(tmp); }
}

public CommitResult Commit(ReviewDecision d, bool loseReceipt)
{
    string decisionPath = PathFor("decision", d.OperationId);
    if (!WriteOnce(decisionPath, d))
    {
        var existing = JsonSerializer.Deserialize<ReviewDecision>(File.ReadAllText(decisionPath))!;
        if (existing.Approved != d.Approved) return new(d.OperationId, "ConflictingDecision", false, EffectCount());
    }
    if (!d.Approved) return new(d.OperationId, "Declined", false, EffectCount());

    bool created = WriteOnce(PathFor("effect", d.OperationId), d);
    if (loseReceipt) throw new LostReceiptException();       // teaching fault: after the effect
    return new(d.OperationId, "Committed", Replay: !created, EffectCount());
}

Flushing and renaming is a reasonable local protocol, not a proof against every failure. It was exercised here against process exit and restart, lost responses and repeated approvals. It was not tested against power loss, and it does not make the file system transactional. If the decision is written and the process dies before the effect, reconciliation reports NoLocalEffectRecorded; resuming the approved operation then creates the one record.

Run the interruption across separate processes

Separate process invocations make lifetimes unambiguous: nothing survives in memory between commands. The sequence below is what the program did on .NET 10 with Microsoft.Agents.AI.Workflows 1.28.0.

The console output, abridged to the lines that matter:

begin:      status=PendingRequests op=op-v1-e18cbda41ce2f1bd... checkpoints=2
approve:    resumed status=PendingRequests requestReemitted=True
            executor failed: LostReceiptException                       (exit 42)
reconcile:  state=Committed effects=1
approve:    resumed status=PendingRequests requestReemitted=True
            result={"Status":"Committed","Replay":true,"EffectCount":1}
approve:    result={"Status":"Committed","Replay":true,"EffectCount":1}

decline:    result={"Status":"Declined","Replay":false,"EffectCount":0}
approve:    result={"Status":"ConflictingDecision","Replay":false,"EffectCount":0}

naive key:  result={"Status":"Committed","Replay":false,"EffectCount":2}

The last line is the control experiment. With the effect file named by a fresh GUID per attempt instead of the operation identity, the same lost-receipt sequence produced two effects. Everything else was identical: same workflow, same checkpoint, same approvals. The idempotency lives entirely in the key.

Pitfalls

• Identity from display JSON. Serializer settings change; framed canonical bytes do not.
• Approving the current proposal. Approval must name the identity it approves. A stale tab must fail loudly.
• Interpreting a missing receipt as failure. The effect may exist. Reconcile before retrying, and report uncertain state as uncertain.
• Reading NewEvents twice. The second read is empty by design. Copy it once.
• Deploying a changed graph under pending checkpoints. Executor IDs and topology must match. Version the workflow and drain or migrate pending cases.
• Sharing a file checkpoint store across instances. It is process-exclusive by design. Use a store built for concurrent access if you need several hosts.

When the effect lives in someone else's system

Replace the local review record with a call to a remote ticketing API and the proof changes. A local file cannot tell you whether the remote side committed before your request timed out. The receiver needs to accept the operation identity as an idempotency key, or offer a lookup by that key, or some other protocol suited to its transaction boundary. A negative answer from an eventually consistent lookup may not settle the question, so keep the state "unknown" visible until evidence resolves it. The same operation identity must survive every retry. The receiver-side half of this problem is worked through in after the receiver commits: recover a lost acknowledgement.

What to rely on for which guarantee

The whole chain, end to end

The series ends with one inspectable chain. A typed question selects a contract. Eligible passages are selected before ranking and must cover every required fact. Specialists are called only where the graph allows, inside a budget, and their findings count only with licensed citations. The proposal binds the exact text, evidence and policy into one identity. A person approves that identity at a pending request that survives a restart, and the effect is keyed so that a lost receipt costs a reconciliation, not a duplicate. Each stage keeps what the next stage needs to check, which is what turns a model's judgment into part of an application with owners.

Checklist

• The operation identity is SHA-256 over length-prefixed UTF-8 fields with documented normalization, checked against vectors from a second implementation.
• Approvals carry the operation identity; stale or omitted approvals are refused.
• Human edits and source-version changes create new proposals with new identities.
• Decisions are immutable; a changed decision for the same identity is a conflict.
• Every side effect is keyed by the operation identity and written once.
• Reconciliation reads the effect store and distinguishes committed, declined, approved without effect, and pending.
• Pending checkpoints record a workflow version, executor IDs are stable, and NewEvents is read once per step.
• The checkpoint directory is private, single-writer storage.

Series: Decision APIs in .NET, part 10 of 10. Previous: Part 9, when an agent coordinator has a real choice. Back to the Decision APIs series overview.