An export you can verify

Article 20 of the GDPR gives a person the right to receive their data. On a hosting platform that is not a small file. An organisation export reads roughly seventy sections, each capped at twenty five thousand rows, and it takes minutes rather than milliseconds.

Until recently ours was built inside a server action: a web request assembled the whole thing in memory and handed it back as one string through the response. That works right up to the account where it does not, and the account where it does not is the one that needs it most.

It is a job, and there is only one way through

An export is now a durable operation with four stages: queued, collecting, packaging, ready. A worker in the gateway picks it up every thirty seconds, locks one row, builds the sections, writes the file, and hands back a token.

Four numbers hold the whole thing together, and each one is a decision rather than a default.

The old synchronous path was not kept as a shortcut for small accounts, and that was a deliberate decision rather than laziness.

Two paths would be two truths. Which one runs would depend on an estimate made in advance, and the estimate that matters here is "is this account small", which is exactly the thing you cannot know before you have counted the sections. Get it wrong and the export falls over on the path that has no safety net.

Two paths would also be two kinds of proof. The audit row, the receipt with a checksum in it, and the list of truncated sections all exist only on the job path. A "small" export would quietly be delivered without any of that, and the question "what did we hand out in March" would have an answer that depends on how large the account was at the time.

The price is one extra click, and that click was going to be there anyway: long running work belongs in the task tray, and that is where it now sits, with a progress bar instead of a browser doing nothing for two minutes.

Where the code had to live

The builders moved out of the web application and into the package that owns database access. That was not tidying. The gateway image copies the gateway and the shared packages and does not copy the web directory, so an import reaching into web/lib compiles locally and is missing in production. It is a measured boundary rather than a stylistic one.

The export operation also has a resource type of account, and it is the first one with no module behind it. Every other durable operation is about a server, an agent, a piece of hardware or an automation, and each of those maps onto a permission module that decides who may see it. An account export maps onto nobody: a member's permission set carries rights, not identity. Visibility is answered by asking whether the viewer is the person who asked for it, and the live event channel returns false for everyone else.

A temporary file, then a rename

The file is written as <name>.part and then renamed into place. A rename inside one directory is atomic, so a reader can never get half a file, and half a file is worse than no file: it has a checksum that does not match and nothing that says why.

The checksum is a sha256 of the bytes, and it goes into the receipt alongside the size. That is what makes the export verifiable rather than merely delivered. Somebody who has the file in a year can still show that it is the file we produced.

The directory is a bind mount that hangs in two containers: read write in the gateway, which writes, and read only in the web application, which serves. Two processes, two roles, one directory.

That mount is easy to forget, and forgetting it produces a very specific and very confusing failure. The task says ready. The download returns 410. Nothing in the log says "wrong directory", because as far as each process is concerned everything worked. The code therefore warns once, loudly, if the directory is not configured, and the fallback is a path next to the repository rather than the system temporary directory: a file in a temporary directory disappears on a restart, and then a row saying "ready to download" points at nothing.

The token is single use, and single use is one statement

The download link carries a token that is valid for twenty four hours, while the file itself stays for seven days.

Both limits are needed. Single use alone would leave a token usable for months as long as nobody clicked it. Time limited alone would make it reusable for a day by anyone who saw the URL: a proxy log, a browser history, a shared mailbox. Whoever misses the window asks for a new export, which takes a minute and produces a fresh, correct copy rather than a link that lingers in an inbox for a week.

The guard that actually enforces single use is not the timestamp. It is one conditional statement: update the row, set the downloaded timestamp, where the downloaded timestamp is still null. Two simultaneous requests with the same token touch exactly one row and the second gets zero back. Checking and then writing would be two statements and therefore two chances for something to slip in between.

The token on its own is also not enough. The caller passes the logged in user and it has to be the person who requested the export, which is why that route runs under the session policy rather than the token policy. And the ownership check comes before every other refusal: somebody who should not have the token should not learn from it whether the export is expired or already collected.

Seven days for the file is short on purpose. The file is a complete copy of everything we hold about a person or an organisation, and while it exists it is a second place where all of it lives. A backup picks it up, and the careful allowlist that decides what goes into an export did its work at the moment the file was written and protects nothing afterwards. It is explicitly not a retention period for the data itself, which sits in the database under its own rules. It is the retention period of a transcript.

Truncation is an open check, not a footnote

Each section stops at twenty five thousand rows. A section that hit that ceiling is named in the envelope and appears in the receipt as an open check: something the reader is expected to act on, next to the outcome rather than buried in it. An export that silently stopped at the cap would be an answer to a data subject request that is incomplete without saying so, and that is worse than a refusal.

Measured with an account carrying twenty five thousand and five rows in one section: the section truncated, the open check appeared, and a second attempt at the same download link returned 410.

One table deliberately left out, with the reason attached

Not everything goes in, and the list of what stays out carries a written justification per table rather than a policy sentence at the top.

The clearest example is the monitoring usage ledger. There are three tables there: the hourly buckets, the entitlement row, and the row per dispatched check underneath them. The first two go in, because what a data subject request wants to know is how much was measured for me and what my account was allowed, and that is exactly what those two say. The third is hundreds of thousands of rows a month carrying the same totals, and it is administration of our dispatching rather than something the customer entered.

That reasoning has a date on it. If that ledger ever carries an invoice line, the omission has to be weighed again, because the evidence row behind a charge is a different kind of record than a shipping log. Writing that down next to the exclusion is the difference between a decision and a habit.

Two other tables sit out for the same reason: the per attempt administration underneath a scheduled measurement, and the scheduling ticks themselves. Both say something about how we executed, and nothing about the person asking.

What it looks like from the outside

Somebody asks for their data. A card appears in the task tray with a progress bar. A few minutes later there is a notification, a download link that works once, and a receipt naming the size, the checksum, and anything that was cut short. Seven days later the file is gone and the administration of what was handed out stays for ninety.

None of that is remarkable, which is the point. The remarkable part is what is missing: no path where the same request produces a different kind of proof, no file that can be read while it is half written, and no link that keeps working after it has been used.

From the platform work on Provibr. Why the operations view reads existing rows and never writes back into them is in projecting, never replacing, and why a progress bar here refuses to predict a finishing time is in a progress bar that refuses to guess.