Small Buffers, Large Files: An Azure Blob Transfer Lab
Throughput gains must be evaluated alongside memory, concurrency, retries and integrity.
THE READING ROOM
THE READING ROOM
Throughput gains must be evaluated alongside memory, concurrency, retries and integrity.
Faster implementation can increase the review queue without increasing accepted throughput.
The dangerous part of a large data job is often a small piece of code that looks beautifully simple: load the records, transform them, save the result. Each verb is reasonable. The problem is the missing boundary around “the records.”
A service can become faster in a benchmark and slower for its users. That sentence sounds like a paradox only if we assume that every performance number describes the same thing.