Skip to content

Serving layer

Entry 007 — Locking down a bucket broke a door that was ruled open

Enforcing public-access-prevention across every bucket was right in general. One bucket was deliberately world-readable, and two live routes read it anonymously on purpose. Both decisions were correct and they could not both hold.

Charles Jones3 min read

Theory — the hypothesis held before the instruments answered

“Enforce public-access-prevention on every bucket. There is no bucket that should be readable by anyone who asks.”

Date — enforced and detected 2026-09-05; ruled the same day; cured, measured and closed 2026-09-05.

Symptom

Two routes on the live site answered 200 and served nothing.

callcharlie.io/vitals.json 200 counts: total 12 · ok 0 · absent 12
callcharlie.io/governance.json 200 "the bucket object answered HTTP 403"
totals null · 0 rulings

The page they feed still rendered twenty rows, because those were baked at build time and the page holds baked values when a live refresh fails. Nothing looked broken from the outside.

Diagnosis

A security pass enforced public-access-prevention on every storage bucket in the project. That is the right default and it was applied correctly.

One of those buckets was deliberately world-readable, by an earlier ruling, because it exists to serve two small JSON files to this website. Both routes read it anonymously on purpose, and each said so in its own header: no service account, no credentials in the runtime … one fewer thing that can silently expire.

Enforcing the default removed the only access path those routes had.

Neither decision was wrong. A general rule met a specific, documented exception, and the two were held by different people at the same time. The routes degraded exactly as they were built to — 200 with a named absence, never a stale copy, never a 404 — which is why nothing paged anyone and why the page still looked populated.

Cure

The routes now read the bucket as the service’s own identity, using a token the platform mints per request. The bucket stayed private; no exemption was taken.

The old headers had argued against exactly this: holding no credential was described as one fewer thing that could silently expire. That argument was about a key — a file to store, rotate and eventually leak. A per-request token is none of those things; it never enters the repository. The headers now say what is true rather than defending a decision that had been overtaken.

One shared reader serves both routes. Two implementations of one read is a shape this estate had already paid for twice that week.

Measured

anonymous GET, both objects 403 the bucket is still private
callcharlie.io/vitals.json 200 · ok 9 · absent 3
callcharlie.io/governance.json 200 · totals populated · 133 rulings
a fresh build baked a snapshot with 9 fields measured, not 0

The three fields still absent are the ones that require a billed query nobody authorised. That is the schema working.

The fix reached production without a production deploy, because those two routes follow the latest container on every channel — a coupling filed as a hazard hours earlier, working in the other direction for once.

Permanent

A bucket whose public-readability is a ruled contract is not an oversight to be corrected by a sweep. Before a general control is applied to everything, the exceptions have to be found — and an exception documented only in the header of the code that depends on it is not findable by the person applying the control.

The safe-looking option was to exempt the one bucket. The harder option was to authenticate the readers and leave the lockdown intact. The harder option is the one that was taken, and the estate ends up with a private bucket and authenticated readers rather than a public bucket and a note explaining why.

Sources

Estate record, private: technical-debt ledger TD-MM-149, opened and closed 2026-09-05; rulings of 2026-09-05 establishing the original public-read contract and the authenticated-read cure.


Two correct decisions, made in different rooms, that could not both be true at once.

Charles Jones

AI Platform Engineer · Marine Engineer

Related articles

navigate openesc close