Skip to content
About
Main content

Target file size

Why is the small-file problem economic before it is a performance problem?

Per-request pricing did the crossover arithmetic for one request. The important shape appears at fleet level. Request cost scales with (files touched × number of queries), while data volume stays constant. The penalty grows in a dimension unrelated to how much data you have.

If the same dataset costs about $2,000 per year to store, the request line exceeds the storage line by about seven times, and nothing in the query profile shows it.

Why the small-file problem is economic before it is a performance problem

Send a thousand GETs at the same time and Little's Law (items in flight = rate × time) hides almost all the per-request . But the thousand fees remain a thousand fees. Concurrency is a latency-hiding device. It has no effect on any metered quantity. That is why cost problems survive every performance fix: the fix removes the visible symptom (slowness) and leaves the invoice unchanged. Which frame applies depends on what you count: summed over a year of a fleet's queries, request fees can outgrow the storage bill; for any one query they are a few cents, and the damage there is planning time (see Why small files appear and Manifest rewriting and orphan files).