The silent bucket dump
The bug that mattered here never threw an error. Every product on the shared content plane gets its own storage bucket now, but that was not always true, and the way it used to fail is the more interesting part: when an editor's account had no explicit mapping to a tenant bucket, the upload did not fail — it quietly landed in one shared default bucket instead, and the only trace was a warning line in a log nobody was watching. A convenient fallback and a silent one turned out to be the same design mistake wearing two different names.
Finding it meant reconstructing, after the fact, which uploads over what period had gone to the wrong place, migrating them to where they should have lived, and only then fixing the actual cause. The fix inverted the failure mode on purpose: the default is still there, because a content system needs somewhere for an unmapped upload to land, but it is loud now — every fallback logs prominently and is treated as something to go fix, not something to quietly tolerate. A silent, convenient default is worse than a noisy, inconvenient one, because the noisy one gets fixed.
The same investigation surfaced a second, related gap. The pipeline that converts an uploaded image also updates the original file's record in the content system, and it turned out that update was quietly deleting the original file it converted from — nobody had asked it to preserve the source, so it did not. Recovery for anything already affected meant going to the nightly backup mirror rather than the live bucket, which worked, but is not where a fix belongs. The pipeline now copies the original into its own protected location before it does anything else to it, so a conversion can never again cost the source.
The structural fix outlasted both individual bugs: every product's storage now derives from one registry entry — a bucket name and a storage location, nothing else — rather than from configuration duplicated per product. Onboarding a new tenant went from a sequence of manual steps someone could forget one of, to one line in a file that everything downstream reads from. By the time the same recipe had been proven once, extending it to every other tenant on the shared plane took a single day, which is the real measure of whether a fix addressed the actual cause or only patched the symptom in front of it.
← all logs