Native Decision API Clients in C#: Wire Shapes and Stalled Bodies

Series: Decision APIs in .NET, part 5 of 10. Previous: Part 4, thresholds from the cost of each error. Next: Part 6, two Azure decision endpoints.

The routing task from Part 3 works against one provider. Someone points it at a second one by changing the base URL, and the first request comes back 422. They fix the payload, and the next surprise is quieter: a response whose confidence field is missing deserialises happily, with a confidence of 0. A week later, a load balancer sends response headers and then stalls, and a request that "has a 100-second timeout" keeps waiting long after those 100 seconds.

None of these are model problems. They are client problems, and they sit in three places: the wire shape of each native protocol, the lifetime of an HTTP response after its headers arrive, and the way failure is reported to the code that has to decide what to do next. This part builds a small C# client for TypeSafe's System One protocol and OpenAI's Decisions API, runs it against a real loopback socket, and compares it with the official OpenAI .NET SDK.

The same three questions travel in two different shapes

The task asks three questions about ticket T-1042: which team should investigate first (a choice over technical, billing, returns and other), whether device, firmware and symptom are all supplied (a predicate), and how severe the service impact is (a three-level score). These are the request bodies the client actually sent, shortened only in the question text.

// POST https://api.typesafe.ai/v1/systemone
{
  "model": "jev-latest",
  "state": "Aurora M4, firmware 2.8; telemetry stopped after the update. ...",
  "questions": {
    "department": { "type": "choice", "instructions": "Which team should investigate first?",
                    "criteria": { "technical": "Device fault or telemetry problem needing diagnosis",
                                  "billing": "...", "returns": "...", "other": "..." } },
    "enough_evidence": { "type": "noul", "instructions": "Are device, firmware and symptom all supplied?" },
    "urgency": { "type": "score", "instructions": "Rate the supported service impact.",
                 "criteria": ["No interruption", "Intermittent with workaround", "Continuous without workaround"] }
  }
}

// POST https://api.openai.com/v1/decisions
{
  "model": "gpt-6-luna",
  "input": "Aurora M4, firmware 2.8; telemetry stopped after the update. ...",
  "questions": [
    { "name": "department", "type": "choice", "instructions": "Which team should investigate first?",
      "choices": [ { "value": "technical", "description": "Device fault or telemetry problem needing diagnosis" },
                   { "value": "billing", "description": "..." }, "..." ] },
    { "name": "enough_evidence", "type": "predicate", "instructions": "Are device, firmware and symptom all supplied?" },
    { "name": "urgency", "type": "score", "instructions": "Rate the supported service impact.",
      "levels": [ { "label": "No interruption" }, { "label": "Intermittent with workaround" },
                  { "label": "Continuous without workaround" } ] }
  ]
}

The semantics match and almost every detail differs. The differences that matter to a client:

Both protocols return a model field that may differ from the alias you sent; TypeSafe's schema says so explicitly. Microsoft-Decision-1 in Foundry uses a providers/microsoft/v1/systemone path and a request "similar to" System One, and documents only the noul, choice and score kinds. Treat it as its own profile that shares this serializer, not as a promise of full compatibility. Part 6 configures it.

Translate at the edge, then validate the meaning

The client has three jobs and keeps them apart. A wire adapter turns the versioned task into a native request and a native response into the answer records from Part 3. The transport owns the HTTP exchange. The validator from Part 3 then checks the normalised answers against the questions that were asked. What does each adapter have to get right?

Four rules fall out of the comparison, and each one is a test in the lab below.

• Identity before value. For System One the map key is the identity; an answer for a name you did not ask is rejected. For OpenAI the position is the identity and the name, when present, must agree with it. A response that puts the urgency answer first, named urgency, is rejected rather than silently read as the department.
• No defaults. A missing confidence is an invalid body. A naive DTO with double Confidence deserialises the same body to a confidence of 0, which downstream code cannot tell from a real zero.
• Check the rubric that came back. System One returns a legend that maps each level to its criterion; OpenAI returns a label with each level. If either disagrees with what was sent, the provider and the application are not talking about the same scale.
• Keep the protocol's vocabulary. Accepting a boolean choice value is a capability this task does not need, so the adapter rejects it explicitly. That is a documented limit of this client, not a claim about the API.

Duplicate JSON properties deserve their own line. {"noul": 0.94, "noul": 0.06} is syntactically valid JSON, and by default both JsonDocument and JsonSerializer accept it and keep the last value: in the lab, a naive read returned 0.06. Since .NET 10, JsonDocumentOptions.AllowDuplicateProperties = false (and the same option on JsonSerializerOptions) makes the parser throw instead. Targets older than .NET 10 can take the option from the System.Text.Json 10 NuGet package, or walk the document and reject repeated names themselves.

Headers arrive before the body, and the timeout stops at the headers

To bound memory, a client reads the status and Content-Length before it buffers anything, which means HttpCompletionOption.ResponseHeadersRead. The documentation is explicit about the side effect: with that option, HttpClient.Timeout applies only up to the end of the headers. Reading the content afterwards is not covered. What happens when the server sends headers and then nothing?

The fix is not a bigger timeout. It is one owner for the whole operation: a linked CancellationTokenSource created per call, passed to SendAsync, ReadAsStreamAsync and every ReadAsync, with the HttpClient's own timeout set to infinite so two clocks cannot disagree. Parsing is synchronous and takes no token, so the client checks the budget just before parsing and bounds the work with a byte limit and a nesting depth instead. For the wider pattern of keeping timeout, cancellation and commit apart, see a timeout is not cancellation.

The client: one budget, bounded reads, strict parse

public async Task<NativeResult> SendAsync(DecisionTask task,
    string deployment, string input, CancellationToken caller)
{
    caller.ThrowIfCancellationRequested();
    var json = wire.Payload(task, deployment, input).ToJsonString();

    using var budget = CancellationTokenSource.CreateLinkedTokenSource(caller);
    budget.CancelAfter(deadline);         // one deadline: headers, body, parse
    try
    {
        using var request = new HttpRequestMessage(HttpMethod.Post,
            new Uri(root, wire.Path))
        { Content = new StringContent(json, Encoding.UTF8, "application/json") };
        using var response = await http.SendAsync(request,
            HttpCompletionOption.ResponseHeadersRead, budget.Token);

        if (!response.IsSuccessStatusCode) return Classify(response.StatusCode);
        if (response.Content.Headers.ContentLength > maxBytes)
            return Invalid("declared body exceeds bound");

        await using var stream =
            await response.Content.ReadAsStreamAsync(budget.Token);
        using var buffer = new MemoryStream();
        byte[] block = new byte[4096]; int read;
        while ((read = await stream.ReadAsync(block, budget.Token)) > 0)
        {
            if (buffer.Length + read > maxBytes)
                return Invalid("body exceeds bound");
            buffer.Write(block, 0, read);
        }

        budget.Token.ThrowIfCancellationRequested();
        using var document = JsonDocument.Parse(
            buffer.GetBuffer().AsMemory(0, (int)buffer.Length), Strict);
        var answers = wire.Read(document.RootElement, task);
        if (DecisionValidator.Validate(task, answers) is { } error)
            return Invalid(error);

        var outcome = answers.Any(a => a is RefusalAnswer)
            ? Outcome.Refused : Outcome.Ok;
        return new(outcome, answers, 200, Raw: document.RootElement.Clone());
    }
    catch (OperationCanceledException) when (caller.IsCancellationRequested)
    { throw; }                            // the caller left: not a provider fault
    catch (OperationCanceledException) when (budget.IsCancellationRequested)
    { return new(Outcome.Deadline, Error: "operation deadline expired"); }
    catch (Exception e) when (e is HttpRequestException or IOException)
    { return new(Outcome.Transport, Error: e.GetType().Name); }
    catch (Exception e) when (e is WireException or JsonException
        or InvalidOperationException or KeyNotFoundException
        or ArgumentException or FormatException)
    { return Invalid(e.Message); }
}

The HttpClient comes from a factory that disables redirects and sets the timeout to infinite. With redirects off, a moved endpoint surfaces as an HttpFailure with its status instead of being followed silently; following a 302 would even turn the POST into a GET. Both Strict options are static: new() { AllowDuplicateProperties = false, MaxDepth = 16 }.

public static HttpClient CreateHttpClient() => new(new SocketsHttpHandler
{
    AllowAutoRedirect = false,                // a moved endpoint is a failure, not a hop
    PooledConnectionLifetime = TimeSpan.FromMinutes(5),
})
{ Timeout = Timeout.InfiniteTimeSpan };       // the client's budget is the only clock

Classify failures so the next decision is possible

An empty route is the worst possible failure report: the caller cannot tell a wrong credential from a refusal from a stalled proxy. The client returns one of ten outcomes, and the categories are deliberately coarse, because a status code alone does not diagnose a cause.

TypeSafe's schema distinguishes 400 (application validation, with a detail) from 422 (request validation, with a list of loc, msg and type). Microsoft's troubleshooting table for Decision-1 maps 404 to a wrong endpoint or path and 422 to an unsupported deployment name or question definition. Both point at configuration, which is why they share an outcome; keep the status and the bounded error detail for the operator, and never log the request's authorisation header.

The client itself does no retries, and in the lab every scenario issued exactly one request. A decision call is an assessment, so retrying it is usually harmless; the action it leads to may not be. If retries are added, one layer owns the attempt count and the total deadline, and an Authentication or InvalidResponse outcome is not retried at all.

Test the stream and the parser against a real socket

Mocking HttpMessageHandler tests the parser but not the stream: it cannot withhold a body, send chunks or break a connection halfway. The lab starts a TcpListener on an ephemeral loopback port, writes raw HTTP/1.1 responses per scenario, records every request, and stops in finally. Run on .NET 10, every scenario produced the expected outcome:

PASS  s1-ok                  Ok: department=technical 0.93; enough_evidence=0.94; urgency=1.1
PASS  oa-ok                  Ok: department=technical 0.93; enough_evidence=0.94; urgency=1.1
PASS  same normalised decision
PASS  raw kept after dispose response model jev-1.14.0, request model jev-latest
PASS  oa-misordered          InvalidResponse (answer 0 is named 'urgency', expected 'department')
PASS  oa-refusal             Refused: department=technical 0.93; enough_evidence=refused; urgency=1.1
PASS  s1-refusal             InvalidResponse (answer type 'refusal' is not valid for 'enough_evidence')
PASS  oa-label               InvalidResponse (level 1 carries label 'Continuous without workaround')
PASS  s1-bad-mean            InvalidResponse (urgency: score 0.5 is not the weighted mean 1.1)
PASS  s1-duplicate           InvalidResponse (Duplicate property 'noul' encountered during deserialization.)
PASS  status-302             HttpFailure 302
PASS  chunked-big            InvalidResponse (body exceeds bound)
PASS  truncated              Transport (HttpIOException)
PASS  stall-body             Deadline (operation deadline expired)  300 ms
PASS  caller cancels during body  TaskCanceledException
PASS  pre-cancelled caller   OperationCanceledException, 0 requests sent

30 passed, 0 failed

The full run has 30 checks, including reordered System One answers, unnamed OpenAI answers, a boolean choice value, a legend mismatch, a missing confidence, malformed JSON, seven status classes, a declared oversized body and stalled headers. The response bodies are authored to the documented shapes; no hosted provider was called. That boundary matters: the lab proves that the transport and adapters behave as specified, not that a deployment exists, that a credential works, or that a model answers the routing question well.

Or use the OpenAI SDK, knowing what it changes

For OpenAI Decisions there is now an official option. Version 2.15.0 of the OpenAI NuGet package adds DecisionsClient, marked experimental (OPENAI001), with typed questions and answers. Pointed at the same loopback server, it parsed the same body:

#pragma warning disable OPENAI001
var client = new DecisionsClient(new ApiKeyCredential(key),
    new DecisionsClientOptions { Endpoint = endpoint });

var options = new DecisionCreationOptions(
    [new DecisionsUserMessageItem([new DecisionsInputTextContentPart(ticketText)])],
    "gpt-6-luna",
    [
        new DecisionsChoiceQuestion(
            [new DecisionsChoice(new DecisionsChoiceValue("technical")), /* ... */],
            [new DecisionsInputTextInstructionsPart("Which team should investigate first?")])
            { Name = "department" },
        new DecisionsPredicateQuestion(
            [new DecisionsInputTextInstructionsPart("Are device, firmware and symptom all supplied?")])
            { Name = "enough_evidence" },
    ]);

DecisionResource decision = (await client.CreateDecisionAsync(options, ct)).Value;
foreach (DecisionsAnswer answer in decision.Answers)
{
    switch (answer)
    {
        case DecisionsChoiceAnswer c: /* c.Choice.StringValue, c.Probabilities[i].Probability */ break;
        case DecisionsPredicateAnswer p: /* p.Probability */ break;
        case DecisionsRefusalAnswer r: /* r.Name */ break;
    }
}

Three things changed, and the lab made each of them visible. The probabilities are float: the SDK's 0.93 widens to the double 0.9300000071525574, harmless under a 0.002 sum tolerance and fatal to an exact-equality test. The SDK sends input as message items and instructions as content parts, a richer form the API also accepts. And its pipeline retries: against a server returning 503, one CreateDecisionAsync call produced four requests before throwing ClientResultException. That is reasonable behaviour, but if your application also retries, the attempts multiply. Set the pipeline's retry policy and network timeout deliberately, and still map the result into the Part 3 records so the validator runs on SDK output too.

Worked example: a naive client and what it hides

var response = await http.PostAsJsonAsync(url, payload, ct);
var dto = await response.Content
    .ReadFromJsonAsync<NaiveResponse>(ct);      // DTO with double fields
var dept = dto!.Answers["department"];
if (dept.Confidence < 0.6) Review(ticket);     // 0 when the field is missing
var evidence = dto.Answers["enough_evidence"].Noul;   // last duplicate wins

var result = await client.SendAsync(task, deployment, ticketText, ct);
switch (result.Outcome)
{
    case Outcome.Ok:
        Route(ticket, result.Answers); break;
    case Outcome.Refused:
    case Outcome.Deadline or Outcome.Transport or Outcome.Temporary:
        HoldForOperator(ticket, result.Outcome); break;
    default:   // Authentication, Authorization, Configuration, InvalidResponse ...
        Alert(result.Outcome, result.Status, result.Error);
        HoldForOperator(ticket, result.Outcome); break;
}

Pitfalls

• Reading answers[0] as the first question you care about. OpenAI returns answers in question order, but match the name when it is present; System One has no order at all.
• Treating Microsoft-Decision-1 as full System One. Same path family, a documented subset of kinds, and "similar" in Microsoft's own words.
• Two clocks. HttpClient.Timeout plus a token deadline gives two different failure shapes for one condition. Keep one owner.
• Catching every OperationCanceledException as a provider failure. It turns users who left into outages and retries.
• Trusting Content-Length as the only bound. Chunked bodies have none; count what you read.
• Renormalising a bad distribution. A sum of 0.7 means something is missing; rescaling hides which.
• Stacking retries. SDK pipeline retries times application retries times gateway retries is a retry storm with extra steps.

Custom client or SDK?

Checklist

• Each protocol has its own adapter, and both produce the Part 3 answer records.
• Answers are matched by name (System One) or by position checked against the name (OpenAI).
• Missing fields, unknown kinds, legend or label mismatches and duplicate keys are invalid responses.
• One linked budget covers headers, body and parse; HttpClient.Timeout is infinite.
• Body size is bounded by declared length and by bytes actually read.
• Caller cancellation is rethrown; an expired budget is Deadline.
• Exactly one layer owns retries, including the SDK pipeline's.
• Tests run against a real socket that can stall, chunk and break.

Series: Previous: Part 4, thresholds from the cost of each error. Next: Part 6, two Azure decision endpoints, one workflow.