Decision APIs: Derive the Threshold From the Cost of Each Error

Series: Decision APIs in .NET, part 4 of 10. Previous: Part 3, designing the decision contract. Next: Part 5, native clients in C#.

The support desk now asks the decision API a second question about ticket T-1042: does this case need urgent specialist handling? The predicate comes back with a probability of 0.20. The team's rule escalates above 0.85, so the ticket joins the routine queue. Nobody can say where 0.85 came from, except that it "felt safe".

Under costs the product owner would actually sign, the same 0.20 should go to a person, and a case at 0.70 should be escalated without one. A threshold is not a measure of caution. It is the point where two expected losses cross, and it moves whenever the cost of an error, the cost of review or the meaning of the probability moves. This part derives the action from those quantities, in C#, and then asks what has to be true about the probability for the derivation to hold.

The threshold follows from the costs, not from a confidence preference

Start by naming the event. Let Y = 1 mean that the case truly needs urgent specialist handling, and let p be the API's estimate of that event for this ticket. This is a new binary question, versioned as urgent-specialist-binary-v1. It is not the ordinal urgency score from Part 3: a mean level of 1.10, or the confidence field next to it, is a different quantity and must never be fed into the equations below.

On the wire the question is a predicate. OpenAI's Decisions API returns {"type": "predicate", "name": "urgent_specialist", "probability": 0.2}; TypeSafe's System One returns the same idea as {"type": "noul", "noul": 0.2}. Neither answer carries a separate confidence field, and neither tells you what to do. OpenAI's guide says it plainly: choose thresholds based on the cost of false positives and false negatives.

So write the costs down. In this illustrative policy a needless escalation costs 1 unit (a specialist's time spent on a routine case) and a missed urgent case costs 8 units. A correct automatic decision costs nothing. The units are deliberately abstract: what matters to the decision is the ratio, and the ratio is a product decision that an engineer should not hide inside a constant.

If p is a usable probability, the expected loss of each action is a straight line in p:

L(escalate | p) = (1 - p) × C_fp = 1 - p
L(routine  | p) =      p  × C_fn = 8p

At p = 0.20, escalation costs 0.8 in expectation and routine handling 1.6. Escalation is the cheaper choice even though urgency is the less likely outcome. That is not a paradox. Picking the more likely class minimises the number of errors, which is the right goal only when every error costs the same.

The lines cross where (1 - p) × 1 = 8p, at p = 1/9 ≈ 0.111. In general the two-action threshold is C_fp / (C_fp + C_fn), the classic result of cost-sensitive decision theory (Elkan, 2001). Every engineer who has typed 0.5 or 0.85 into a config file has made a claim about that ratio, usually without knowing it.

Review is a third action, with a fee and its own mistakes

Most pipelines have a third option: send the case to a person. The usual shortcut treats review as a fixed fee that always produces the right answer. That is a fine first approximation, but it quietly becomes a fact, and then nobody notices that the review queue is where the hard cases go to be misjudged.

Model review honestly instead. Suppose a review costs 0.20 units of effort. Of the routine cases sent to review, the reviewer escalates 5% unnecessarily. Of the urgent cases, the reviewer misses 10%. These are assumed rates for this example, conditional on the kind of case the policy sends to review:

L(review | p) = C_review
              + (1 - p) × rfp × C_fp      reviewer escalates a routine case
              +      p  × rfn × C_fn      reviewer misses an urgent case

              = 0.20 + (1 - p) × 0.05 × 1 + p × 0.10 × 8
              = 0.25 + 0.75p

Review loss rises with p, because the expensive reviewer error happens on urgent cases. At p = 0.20 it is 0.40, which beats escalation (0.80) and routine handling (1.60), so the case goes to review. At p = 0.70, escalation costs 0.30, review 0.775 and routine handling 5.60. Sending that case to review because 0.70 is below a familiar 0.85 would contradict the costs the team just wrote down.

Three lines and one lower envelope: where review wins

The policy picks the cheapest of three lines at every p. Routine handling and escalation form a tent that peaks at p = 1/9. Review is one more line, so it is cheapest on a single interval below the tent, or nowhere. With the default assumptions the interval ends are:

routine = review:   8p = 0.25 + 0.75p     →  p = 1/29 ≈ 0.0345
review = escalate:  0.25 + 0.75p = 1 - p  →  p = 3/7  ≈ 0.4286

Below 1/29 the case is handled routinely, between the two values it is reviewed, and above 3/7 it is escalated. How sensitive are those numbers to the review fee? Move the slider.

The vanishing point has a closed form. Review can only pay off at the peak of the tent, p = 1/9, where the best automatic action costs 8/9. A free review would still cost 0.133 there in expected reviewer errors, so any fee above 8/9 - 0.133 = 34/45 ≈ 0.756 makes review dominated everywhere. If your real review cost is near that number, the honest design may be no review queue at all, or a better review screen that lowers rfn.

The policy in C#: costs in, action and losses out

The code is short because the mathematics is. What matters is that every assumption is a parameter, validated, and returned alongside the action, so that nobody downstream has to guess which costs produced a decision.

public sealed record Costs(
    double FalsePositive, double FalseNegative, double Review,
    double ReviewerFalsePositive, double ReviewerFalseNegative);

public enum ActionKind { Negative, Review, Positive }
public sealed record Losses(
    double Negative, double Review, double Positive, ActionKind Action);

public static class LossPolicy
{
    private const double TieTolerance = 1e-12;

    public static Losses At(double p, Costs c)
    {
        if (!double.IsFinite(p) || p < 0 || p > 1)
            throw new ArgumentException("p must be in [0, 1].", nameof(p));
        Validate(c); // error costs > 0, review >= 0, rates in [0, 1]

        double negative = p * c.FalseNegative;       // routine misses urgency
        double positive = (1 - p) * c.FalsePositive; // needless escalation
        double review = c.Review
            + (1 - p) * c.ReviewerFalsePositive * c.FalsePositive
            + p * c.ReviewerFalseNegative * c.FalseNegative;

        double best = Math.Min(negative, Math.Min(review, positive));
        ActionKind action =
            review <= best + TieTolerance ? ActionKind.Review
          : negative <= best + TieTolerance ? ActionKind.Negative
          : ActionKind.Positive;
        return new(negative, review, positive, action);
    }

    // Review loss is linear in p: it wins on one interval or nowhere.
    public static (double Low, double High)? ReviewBand(Costs c)
    {
        Validate(c);
        double fp = c.FalsePositive, fn = c.FalseNegative;
        double rfp = c.ReviewerFalsePositive, rfn = c.ReviewerFalseNegative;
        double low = (c.Review + rfp * fp) / (fn * (1 - rfn) + rfp * fp);
        double high = (fp * (1 - rfp) - c.Review) / (fp * (1 - rfp) + rfn * fn);
        return low < high ? (Math.Max(0, low), Math.Min(1, high)) : null;
    }

    public static double ReviewBenefit(double p, Costs c)
    {
        Losses l = At(p, c);
        return Math.Min(l.Negative, l.Positive) - l.Review;
    }
}

Run against the default costs on .NET 10, the policy prints the numbers used in this article:

p=0.20 routine=1.600 review=0.400 escalate=0.800 -> Review
p=0.70 routine=5.600 review=0.775 escalate=0.300 -> Positive
band [0.0345, 0.4286]  1/29=0.0345 3/7=0.4286
two-action point 0.1111
c=0.80 review never cheapest
review vanishes at c = 0.7556

With finite reviewers, review the largest avoidable loss

The three-action rule treats every case independently. A real review team is finite. Suppose the five illustrative tickets below all land in the review band, and only two reviewers are free this hour. Which two should they take?

The instinctive answer is "the most uncertain ones", meaning the probabilities closest to 0.5. Under symmetric costs that is right. Under these costs it is not, because the benefit of a review is the gap between the best automatic loss and the review loss, and that gap peaks at the top of the tent, near 1/9.

Apply that to the queue. The five reviewed tickets are T07 (p = 0.30), T08 (0.20), T09 (0.10), T10 (0.08) and T11 (0.05). Ranking by closeness to 0.5 picks T07 and T08. Ranking by review benefit picks T09 and T08. Sending the other three to their best automatic action, the expected loss over the five cases is 2.715 units for the uncertainty ranking and 2.465 units for the benefit ranking. Same reviewers, same probabilities, 9% less expected loss.

Two cautions. First, a capacity limit changes the policy, so name it: "review the top k by benefit per hour" is a different rule from "review everything between 1/29 and 3/7", with different outcomes to evaluate. Do not quietly lower the threshold until the queue fits and call it the same policy. Second, queueing changes the review itself. A reviewer with forty cases waiting makes different mistakes from one with four, so capacity and reviewer error rates are not independent assumptions.

Every threshold assumes the probability means what it says

All of the above treats p as a calibrated probability: among many cases scored near 0.2, about 20% are truly urgent. A model can rank cases well without that property. Guo and colleagues showed that modern neural networks are often overconfident in exactly this sense, and that a simple post-processing step, temperature scaling, fixes much of it on held-out data. Their result is a reason to check, not evidence that any particular decision endpoint is or is not calibrated on your tickets.

The providers say the same in their own words. Microsoft's guide for Microsoft-Decision-1 notes that calibration is strongest on familiar task types, and asks you to validate on data representative of your use case. A field named confidence is a separate thing again: TypeSafe documents it as a statistic computed from the probability distribution of a choice or score answer, not as the probability that the selected label is correct.

Two families of metric answer two different questions. Discrimination asks whether urgent cases get higher scores than routine ones; AUC is the share of urgent and routine pairs that are ordered correctly. Calibration asks whether the numbers match frequencies. A threshold policy needs the second, because 1/29 and 3/7 are positions on the probability scale.

The Brier score measures both at once. For one case it is (p - y)²; for a set, the mean. It is a proper scoring rule: if the true probability for a given information state is q and you report p, the expected score is

q(1 - p)² + (1 - q)p²  =  (p - q)² + q(1 - q)

The second term does not depend on what you report, and the first is zero only when p = q. Honest probabilities minimise the expected score, so a model cannot improve its Brier score by being bolder than the evidence. Four rows make the effect visible: forecasts 0.9 (urgent), 0.9 (routine), 0.6 (urgent) and 0.2 (routine) score 0.01, 0.81, 0.16 and 0.04, a mean of 0.255. The single confidently wrong row contributes almost 80% of it.

A monotone transform keeps the ranking and moves cases across thresholds

To see discrimination and calibration separate, take twelve illustrative test rows and transform every probability with p' = p² / (p² + (1 - p)²). That formula looks arbitrary until you write it in log-odds: logit(p') = 2 × logit(p). It is temperature scaling with a temperature of 0.5, the same family Guo and colleagues used to calibrate, here pushed in the overconfident direction on purpose.

The full comparison, computed by the same program:

The worse probabilities produced the lower sample loss. That is not a contradiction, and it is not a reason to sharpen. The 0.75 units saved came from three routine tickets that happened to stay routine. Had any one of them been urgent, routine handling would have cost 8 units where review would have cost 1.00, wiping out the 0.75 saved nine times over. The Brier score penalises exactly that exposure; a twelve-row policy loss cannot see it. Choosing the transform because it won on these rows would turn the test set into part of model selection, and the next twelve rows would be the real test.

The table also separates what happened from what was assumed. Automatic errors are realised: the labels are known, so T02 and T05 cost 1 unit each. Review outcomes are not observed in this data, so each reviewed case carries its expected loss under the reviewer model: 0.20 + 0.10 × 8 = 1.00 for an urgent case and 0.20 + 0.05 × 1 = 0.25 for a routine one. Keep the two columns apart in any report. Blending them into one number hides which half of the result is evidence and which half is assumption.

Six correct decisions do not establish perfection

Small evaluation sets are where these mistakes become decisions. An automation rule that handled six cases with no error is often reported as 100%. Under a binomial model, the 95% Wilson interval for 6 of 6 is 61% to 100%; for 60 of 60 it is 94% to 100%. Same observed proportion, very different evidence. In the table above, "5 of 7 automatic decisions correct" has an interval of roughly 36% to 92%, which is to say almost no information about the automatic error rate.

Reliability plots, the standard picture of calibration, suffer the same problem. A plot that bins twelve rows into five probability ranges puts one or two labels in some bins, and a dramatic-looking deviation can be a single ticket. Print the count next to every point, add an interval, and do not draw a smooth line through sparse bins. Calibration can also differ by slice: language, device family or missing evidence. Check the slices where the action matters, as described in RAG quality as a distribution, not a single score.

Keep three activities on three separate sets of rows. Selecting a threshold or a capacity rule from observed policy loss is one. Fitting a calibrator, such as a temperature, is another. Evaluating the result is the third. Related tickets (the same incident, a translated copy, a duplicate) belong together on one side of every split; an ID check cannot detect that.

Worked example: from a magic threshold to a stated policy

// "0.85 feels safe"
var action = answer.Probability >= 0.85
    ? ActionKind.Positive
    : ActionKind.Negative;

var costs = new Costs(FalsePositive: 1, FalseNegative: 8, Review: 0.20,
                      ReviewerFalsePositive: 0.05, ReviewerFalseNegative: 0.10);

if (question.Id != "urgent-specialist-binary-v1")
    throw new InvalidOperationException("Policy expects the urgency predicate.");

Losses l = LossPolicy.At(answer.Probability, costs);
log.LogInformation("{Ticket} p={P} {Action} losses={Neg}/{Rev}/{Pos} costs={@Costs}",
    ticket.Id, answer.Probability, l.Action, l.Negative, l.Review, l.Positive, costs);

The question check is not ceremony. If someone passes the ordinal urgency mean, or a choice answer's confidence, into this policy, the arithmetic still runs and the decisions are meaningless. Bind the policy to the question it was derived for. Downstream, the action itself still needs authority: escalating a ticket is cheap to undo, issuing a refund is not, and Part 10 treats the reviewed action and its idempotency separately.

Pitfalls

• Thresholding the wrong number. A score mean, a choice confidence or a provider field with a reassuring name is not the probability of your event. Check the question and the field.
• Review as a free oracle. A review loss without reviewer error rates overstates the value of review exactly where costs are most asymmetric.
• Prioritising by uncertainty under asymmetric costs. Rank a constrained queue by avoidable loss, not by distance from 0.5.
• Reading AUC as calibration. Any strictly increasing transform leaves AUC unchanged and can move every case across your thresholds.
• Selecting on the test set. A transform, threshold or temperature chosen because it won on the evaluation rows has not been evaluated.
• Percentages without counts. Report 6/6 with its interval, not 100%.
• Silent drift of the candidate set or question. A probability from urgent-specialist-binary-v1 does not carry over to a reworded v2; re-derive and re-evaluate.

Which number to use for which decision

Checklist

• The event behind every thresholded probability is named and versioned, and the policy rejects any other question.
• False-positive and false-negative costs are written down, owned by the product, and stored with every decision.
• Review has a fee and conditional error rates, measured on the cases the policy actually sends to review.
• The review band is computed, not chosen, and you know the review cost at which it disappears.
• A capacity-limited queue is ranked by avoidable loss and documented as a separate policy.
• Calibration is checked with a proper score and a reliability view per slice, with counts and intervals.
• Threshold selection, calibration fitting and evaluation use separate rows, grouped by incident.

Series: Previous: Part 3, designing the decision contract. Next: Part 5, native Decision API clients in C#.