Telemetry Event Formats: Bytes, CPU and Schema Compatibility

A telemetry event has several costs before anyone learns from it. The application constructs it, the serializer encodes it, the transport carries it, the receiver decodes it, and some storage system retains enough information to answer a future question. Making one representation smaller can improve one of those stages while making another more expensive.

That sounds obvious until a comparison reduces the decision to the number of bytes in a screenshot. A binary record can look decisively better than JSON when only the uncompressed payload is counted. Compression can change the ordering. Validation can change the CPU cost. A compact record without a clear version or a compatible reader can be cheaper to transmit and more expensive to operate.

This article builds a deliberately small format laboratory. It implements a readable JSON envelope, an application-owned binary envelope, and a Brotli-compressed variant of each. It verifies that all four preserve the selected records, rejects explicit malformed inputs, and measures encoding and decoding separately. The results are local observations from a .NET program, not a benchmark of an OTLP collector, a network or an observability platform.

The goal is to understand what each representation makes us responsible for. A format is not merely a bag of bytes. It is an agreement about interpretation, limits, change and failure. Performance becomes useful when that agreement is visible enough to test.

Begin with the information that must survive

01 / See the idea

One record, several representations

Enlarge diagram

The laboratory record contains a signed 64-bit identifier, a signed 64-bit timestamp field, a finite double value and a label. The timestamp is an integer in this fixture; the schema does not silently infer its unit or timezone. A real telemetry contract must define those semantics. The experiment holds the record meaning fixed so that a smaller encoding cannot win by dropping a field.

Labels are UTF-8 strings limited to 256 encoded bytes. A batch contains at most 10,000 records, and the serialized uncompressed body is limited to 4 MiB. These are application choices for the laboratory, not claims about a standard protocol. They provide a finite domain in which the reader can check lengths before allocating or walking untrusted data.

The input fixtures cover one record, 128 records with a repeated label and 128 records with distinct labels. They differ in redundancy as well as count. The repeated-label case can give compression opportunities that a single record does not. The diverse case keeps the format the same while changing the data's structure.

The correctness fixtures also include an empty batch and a Unicode label. A representation that loses a non-ASCII character is not a more efficient encoding of the same record. A comparison must establish fidelity before discussing bytes or nanoseconds.

The data is synthetic and generated by the supplied code. It contains no production traces, customer identifiers, prompts or private resource names. That makes the archive safe to inspect and the input reproducible without requiring access to an observability account.

A readable baseline deserves a fair contract

The JSON variant serializes a versioned envelope with camel-case field names. It uses System.Text.Json with unknown-member rejection and required-constructor-parameter handling enabled. Before deserialization, it checks for duplicate field names. This is stricter than simply deserializing into an object and accepting whatever value survives a duplicate key.

The duplicate check is a real cost in the measured decoder. It walks the token stream and tracks names within each object before the ordinary deserializer constructs the result. The comparison therefore includes the work needed by this implementation to enforce its chosen contract. A decoder that silently accepted a repeated field would answer a different question.

Here is the relevant configuration:

private static readonly JsonSerializerOptions Json = new()
{
    PropertyNamingPolicy = JsonNamingPolicy.CamelCase,
    UnmappedMemberHandling = JsonUnmappedMemberHandling.Disallow,
    RespectRequiredConstructorParameters = true
};

The envelope carries version 1. The decoder requires that version and the records collection. Each decoded record is validated again for label size and a finite numeric value. Encoding also validates the source records, because producing a payload that the matching decoder must reject is not a successful round trip.

This baseline does not use a source-generated serialization context. That is an obvious candidate for a later experiment, but it is not present in the accepted comparison. The article does not claim that its ordinary serializer is the fastest possible JSON implementation. It is a concrete, inspectable implementation with a specific validation policy.

Readability remains useful. Field names make captures easier to examine, and existing tools can display the envelope. Those benefits do not make JSON free. They explain why a decision about format should include operational use rather than only serialization throughput.

The binary envelope makes more decisions explicit

02 / See the idea

Binary v1 has explicit boundaries

Enlarge diagram

The custom binary representation begins with a one-byte version and a 32-bit record count. Each record then contains the identifier, timestamp, double value, a 16-bit UTF-8 label length and the label bytes. The complete layout and byte order follow the actual BinaryWriter operations in the source. This is an application format for the example, not a replacement standard for OpenTelemetry or another ecosystem protocol.

The count is checked before creating the result array. Each label length is checked against the application limit and remaining input. The decoder rejects a truncated field, an unsupported version and trailing bytes after the declared records. Strict UTF-8 decoding refuses malformed byte sequences instead of silently replacing them with another string.

The encoder's core loop is intentionally conventional:

writer.Write((byte)1);
writer.Write(rows.Length);
foreach (var row in rows)
{
    writer.Write(row.Id);
    writer.Write(row.Timestamp);
    writer.Write(row.Value);
    byte[] label = Utf8.GetBytes(row.Label);
    writer.Write((ushort)label.Length);
    writer.Write(label);
}

There is room to optimize this code. It allocates label bytes and an output array. A span-based writer could change that behavior. The present implementation is not advertised as zero-allocation, and the experiment does not hide these allocations outside the measured encode call.

A literal golden vector provides one independent check of the layout. For a single record with identifier 1, timestamp 2, value 0 and label A, the test compares the encoded bytes with a fixed hexadecimal sequence written independently of the decoder. Round-trip agreement alone would be weaker: an encoder and decoder could share the same accidental layout mistake and still agree with each other.

That golden vector does not establish compatibility with every future reader or language. It establishes a small anchor for this version's byte contract. A real cross-language deployment should maintain corresponding vectors and test both acceptance and refusal behavior as part of its compatibility policy.

Compression changes the boundary, not just the size

03 / Watch the mechanism

A limit must survive every representation

Enlarge diagram

Static view. Play to see how the mechanism unfolds.

The compressed variants first produce the same JSON or binary body, then compress it using Brotli with CompressionLevel.Fastest. The choice favors a particular tradeoff in the local library. It is not a universal setting for telemetry traffic or a claim that another compression level would have the same result.

Decompression reads into a bounded output stream. The code stops if the uncompressed body would exceed4MiB. The bound matters because a small compressed input can expand into much more data. Checking only the compressed length would protect the wrong resource.

Preparation exposed a defect in the initial encoder: it enforced record and label limits but did not enforce the final serialized-body limit. JSON escaping can expand a label substantially. A bounded fixture with 3,000 records containing control characters could therefore satisfy the individual field limits while producing an oversized JSON body. The encoder was corrected to check the serialized body before returning or compressing it, and two targeted rejection cases were added.

That repair illustrates a broader principle. Limits belong to representations and transitions, not only to business objects. A record that fits in memory under one representation may exceed an encoded-message limit after escaping, framing or metadata is added. A valid format implementation must define what happens at that boundary.

The accepted measurements were performed after the correction. The changed length check is part of the code that produced the results. The earlier checks are retained as history, but they are not used to imply that the first implementation satisfied a limit it had not yet enforced.

Correctness asks more than whether decoding succeeds

The final laboratory reports 33 checks. They include round trips of selected normal, empty and Unicode inputs across all four variants, malformed binary cases, unsupported format and schema values, duplicate and missing JSON fields, non-finite source values, the literal binary vector and the oversized escaped-body cases.

The measured path also verifies outputs. After an encoding sample, the last produced payload is decoded and compared with the expected input records. After a decoding sample, the final result is compared again. The loops retain output sizes or record counts in a sink so that the work has an observable result.

These checks are not a security certification or an exhaustive fuzzer. For example, the fixture does not explore every malformed compressed stream or prove resistance to all resource-exhaustion strategies. The explicit maximum body and selected rejection cases make the tested boundaries concrete, but an exposed ingestion endpoint needs a broader threat and operational model.

The four formats also do not have identical error messages. The requirement here is that unsupported or malformed input is refused before being treated as a valid batch. Applications that depend on stable error codes should add an application-owned error contract rather than exposing arbitrary parser exceptions to callers.

It is useful to separate those questions from speed. A fast implementation that silently accepts a duplicate version field cannot be substituted for a strict one merely because both decode the happy-path fixture. Compatibility includes refusal behavior when that behavior protects the meaning of the data.

Measure encoding and decoding as different operations

04 / See the idea

Compression can make a tiny record larger

Enlarge diagram

The actual run used .NET 10.0.15 with SDK 10.0.400 on the shared Windows development host. Each format/fixture combination received ten warm-up encode/decode pairs and six measured samples. A sample repeats 1000 operations for the single-record case or 100 operations for a 128-record batch.

Encoding starts with prebuilt records. Decoding starts with a prebuilt encoded payload. Each timed region includes the relevant validation, serialization or decompression and the materialized output. It excludes input generation, network transport, storage and the later correctness comparison. Allocations are observed using the current-thread managed-allocation counter because these paths are synchronous.

The format order is fixed within each fixture, and all samples execute in one process. That makes the run convenient to reproduce but limits timing conclusions. Tiering, cache state, pool reuse and unrelated host activity can influence later cases. The raw data preserves every sample; the table below is a descriptive summary, not a confidence interval or a production throughput guarantee.

Fixture	Format	Encoded bytes	Median encode	Median decode
one	json	87	1.846 us	8.408 us
one	binary	40	0.477 us	0.484 us
one	json-brotli	91	7.384 us	10.099 us
one	binary-brotli	44	5.792 us	2.437 us
repeated128	json	8920	117.341 us	659.112 us
repeated128	binary	4485	48.871 us	36.117 us
repeated128	json-brotli	1064	162.321 us	490.342 us
repeated128	binary-brotli	806	68.240 us	55.075 us
diverse128	json	10712	178.818 us	445.257 us
diverse128	binary	5765	12.207 us	10.368 us
diverse128	json-brotli	1384	84.828 us	214.072 us
diverse128	binary-brotli	1190	35.782 us	31.154 us

The first result is a useful counterexample to blanket compression advice. For the one-record fixture, compressed JSON is 91 bytes versus 87 uncompressed, and compressed binary is 44 bytes versus 40 uncompressed. The compressed representation is larger. A compression layer has its own framing and cannot manufacture useful redundancy from every tiny input.

For the 128-record fixtures, both compressed variants are substantially smaller than their uncompressed counterparts. The size difference depends on the data: repeated labels and diverse labels produce different outputs. These are exact encoded byte lengths for the accepted fixture, not estimates of network savings after HTTP, TLS, batching or collector metadata.

The timing columns should be read with the fixed-order caveat. Some later, more complicated cases have lower observed times than earlier simpler ones. The run does not establish that adding decompression inherently accelerates parsing. It shows why a short in-process comparison cannot support a complete model of runtime cost merely by sorting medians.

The binary path is smaller than this JSON baseline for the tested records, but that is only one dimension of the choice. It requires maintaining the layout, its version, a compatible reader and a policy for unknown values. If the receiving ecosystem expects JSON or OTLP, introducing this custom envelope may create more conversion and operational work than it removes.

A smaller payload can move work somewhere else

Suppose the application changes its event format to reduce bytes sent to a collector. The receiving side now needs another decoder. If the collector stores a transformed representation, the smaller wire format may not reduce retained storage by the same proportion. If engineers frequently inspect payloads during incidents, specialized tooling may become part of the maintenance cost.

The reverse can also be true. A clearly defined compact format may fit a constrained internal path and remove significant transport or storage work. The point is not to prefer readability at any cost. It is to identify the path where the compact representation has a measurable benefit and account for the work needed to operate it.

Batching is another separate decision. The comparison demonstrates that repetition can help compression, but it does not decide how long the application should wait to collect a batch. Waiting can reduce overhead while increasing the delay before evidence becomes visible. A batch that is lost or partially accepted creates its own recovery question.

A format experiment therefore belongs beside a lifecycle model: when an event is created, when it becomes eligible for export, when its bytes are accepted and when the observation can be queried. The payload size influences that system, but it does not define all its outcomes.

The distinction is especially important for AI telemetry. A prompt or response body may dominate byte volume, yet including it can create sensitivity and retention concerns that no serializer benchmark resolves. A carefully selected structural event may be more useful than compressing a much larger body that was never needed for the operational question.

Versioning is part of the performance contract

05 / See the idea

A faster decoder can accept the wrong contract

Enlarge diagram

The custom envelope has one supported version. A reader rejects another version instead of guessing its layout. That is intentionally conservative. A real evolution strategy may support multiple versions, optional fields or an explicit conversion step, but those choices add code and must be represented in the comparison.

The JSON variant also has a version, even though its field names look self-describing. A field can keep the same name while changing its meaning. A duration can move from milliseconds to seconds; a completion count can move from model responses to independently accepted outcomes. Human-readable syntax cannot prevent semantic drift.

Golden vectors and fixture corpora help make compatibility reviewable. They should include old valid records, new valid records where supported, and inputs that must remain invalid. A proposed optimization should preserve the declared contract, or announce a new contract explicitly. Silent changes are particularly dangerous in telemetry because the output often continues to look plausible.

The experiment's ordinary limits are versioned choices too. Increasing the maximum label size or allowing nested JSON changes resource behavior. Removing duplicate-field rejection changes refusal behavior. Those modifications may be sensible, but they should not be smuggled into an optimization whose reported result claims to measure the same work.

A useful result package therefore includes more than a chart. The reader needs the source, configuration, input generator, literal layout example, raw samples and the exact conditions that make a comparison valid. Without them, a smaller number can hide a smaller promise.

Choose the representation for the boundary you actually own

Download the complete codec laboratory and raw results. The code is small enough to inspect and has no external service dependency. It does not require private telemetry or an account. Run the correctness mode before reproducing the successful-path measurements.

The next useful experiment depends on the decision. A source-generated JSON variant could test a more specialized baseline. A streaming writer could examine output-buffer ownership. A transport test could include batching and delivery. A compatibility suite in another language could test the binary layout outside the current implementation. None of these results is implied by the present run.

The lesson I would carry into a production review is that format selection joins semantics and mechanics. Count bytes, but say which bytes. Measure CPU, but identify the work and configuration. Measure allocation, but do not confuse it with retained memory. Preserve the validation that makes the result meaningful, even when removing that validation would make the benchmark faster.

A format earns its place when another engineer can explain both its benefit and its obligations. The best result may be to keep JSON for a public boundary, use compression for larger batches, and reserve a compact versioned representation for one internal path. That is a more useful engineering conclusion than declaring one format the winner everywhere.

Read the configuration as part of the experiment

Several API options in this example affect correctness rather than merely speed. RespectRequiredConstructorParameters changes whether a missing non-optional constructor argument is accepted during deserialization. The laboratory turns it on explicitly because the envelope's version and record collection are required. A code sample that omitted the setting could still compile while accepting a different set of inputs. Microsoft's required-properties documentation.

UnmappedMemberHandling.Disallow supplies a different rule: a JSON member without a matching contract member causes rejection. It does not replace the required-member rule, and neither setting is used here as a substitute for the explicit duplicate-field scan. These are three separate questions: did everything required arrive, did anything unsupported arrive, and was a name repeated? Microsoft's unmapped-member guidance.

That separation is a useful review technique. When an implementation says it validates JSON, ask which invalid cases are rejected and which configuration establishes each behavior. A serializer option should be connected to an expectation in the fixture, rather than treated as a reassuring word in a code listing. The companion includes the selected malformed inputs so that a later change can be checked against the same promise.

The binary writer has its own trap for casual review. Calling Write(string) would introduce BinaryWriter's own string-length convention. This format instead encodes the label to UTF-8 bytes, writes an explicit 16-bit byte count, then writes those bytes. Replacing that sequence with a convenient string overload would change the wire representation even though the source code might look equivalent to a reader. The golden vector is designed to make such a change visible. BinaryWriter overload reference.

Compression also has an ownership boundary. Disposing the Brotli encoder finalizes the encoded stream before the companion takes the output array. The outer memory stream remains open for that operation. On decode, the companion reads until the compressed stream reports completion while enforcing its uncompressed-body bound. Those lifecycle decisions are part of producing and consuming a complete representation, not incidental cleanup after a benchmark. BrotliStream reference.

None of these explanations requires inventing a new optimization. They demonstrate why a performance review should read the whole operation, including setup that is inside the boundary and lifecycle work that cannot be omitted. A benchmark that takes the output before finalization or compares differently validated inputs can report precise numbers about incompatible operations.

A useful exercise is to change one rule in a separate branch. Permit unknown JSON fields, replace the duplicate scan, use a different compression level or change the label-length representation. Before looking at timing, list the compatibility and failure expectations that changed. If the expected set is different, report the result as a different contract or an explicitly reviewed tradeoff.

This discipline makes the laboratory more reusable than a single winner chart. Readers can substitute a more suitable serializer, another language implementation or a streaming writer while retaining the independent fixtures and known limits. They can also discover that the new implementation requires different limits. The evidence then records a decision about meaning and resources, not simply a contest over one elapsed-time number.

For reproduction, the project has no external NuGet dependency. The local global.json selects SDK10.0.400, and the project requests runtime10.0.15 with roll-forward disabled. Run from the lab directory so that SDK selection uses that file. Execute dotnet run -c Release -- check, then run the built executable's measure mode and preserve its JSON output.

The report contains every sample, including managed allocation for encode and decode. The article's time table does not replace those raw values. If a later run uses another runtime, processor or serializer configuration, retain both reports with their conditions rather than silently replacing the original observation. A successful reproduction means the code and contract can be executed and inspected; it does not require another machine to produce the same nanosecond value.

Source code and execution evidence

Download the complete lab, source record and raw execution results. This package supports the specific execution claims in this article. It is not a production certification.