DevInside

Agent Reliability Field Guide

Five practical questions about AI tool authority, retries, retrieval, human approval and observable effects.

A compact guide to five decisions that determine whether an AI workflow is safe to rely on. Each question links to a public engineering lab. The examples show specific mechanisms; they do not certify an arbitrary product or replace testing in its actual environment.

1. Where does the authority come from?

A model-generated request is not proof of permission. Identify the caller through a trusted boundary and enforce the operation against that identity. Check the record, tenant and action, not only whether the tool name is allowed.

Review question: if the model changes the customer or tenant in its arguments, what prevents access to someone else's data?

Inspect the .NET tool and stock-reservation example →

2. What does a lost response mean?

An effect can succeed before a timeout. Treat an unknown outcome as a state that needs reconciliation. A retry must refer to the same logical action and the receiver must prevent duplicate effects where the operation requires it.

Review question: what durable evidence lets the next process distinguish “not executed” from “executed, but the receipt was lost”?

Follow restart, receipts and reconciliation →

3. Does the retrieved evidence license this answer?

Similarity is only one part of retrieval. A source must be eligible for this caller and question, represent the right version, and contain the exceptions that change the conclusion. A genuine citation can still accompany an unsupported instruction.

Review question: can the evaluation fail a plausible answer that omits an exception, uses the wrong scope or relies on an inaccessible source?

Work through eligibility, versions and exceptions →

4. What exactly did the person approve?

Bind approval to the proposal the reviewer actually saw, including the relevant content and policy. If that proposal changes, the system needs to decide whether the previous approval still applies. Resume and recovery must preserve the same meaning.

Review question: can you change the recipient, content or effect after approval and still execute under the old decision?

Inspect approval after a restart →

5. Which observation proves what happened?

A successful response, a read-only label or an agent's summary is not a complete account of side effects. Observe the state and boundary relevant to the claim. Keep the conclusion as narrow as the evidence.

Review question: which persisted record, trace or independently checked outcome would contradict the tool's account?

Investigate a read-only claim at the database boundary →

Use this in your next review

  1. Choose one workflow and one consequential action or answer.
  2. Write down the caller, permitted scope and expected outcome.
  3. Choose one failure from the questions above and reproduce it in an authorised test environment.
  4. Record what actually happened and what remains unknown.
  5. Agree the next change and the observation that would show it worked.

The code and labs remain public. Field Notes will connect these investigations in a short email series. For a review of your own workflow, see the AI workflow review.