Sixteen thousand keys, nine languages

Our English message catalogue currently holds a little over sixteen thousand strings. There are nine such catalogues, one per language, and they are required to have exactly the same keys. That is somewhere north of a hundred and forty thousand pieces of text that all have to stay true to each other while a dozen people change the product underneath them.

The interesting thing about that problem is not the volume. It is that almost none of the failures announce themselves. A translation with a typo gets reported by a customer within a week. A translation that quietly dropped an argument renders a sentence with a hole in it, forever, and nobody files a ticket about a sentence that reads fine.

This is the set of guards we ended up with, and the reasoning behind each. They are not specific to any framework; the failure modes are a property of having more than one language.

The class of bug that matters

Start with the one that made us take this seriously.

An English string says Welcome back, {name}. Somebody translates it, and the translation is Welcome back. It is grammatical, it is a reasonable sentence, and it renders without an error in any templating system in existence. The parameter simply does not appear. There is no exception, no log line, no missing key warning.

That bug is invisible to every test that checks whether keys exist, which is the test almost everyone writes first. The only thing that finds it is a comparison: extract the set of placeholder names from the reference string, extract them from the translation, and require them to match.

Once you have built that comparison, three refinements turn out to be necessary, and each of them came from a real string that slipped through.

Compare argument types, not just names. A bare {count} does not format. It prints the raw number, with no thousands separator, in any language. Write {count, number} and it formats. If the type is present in English and absent in the translation, the name matches, the branch structure matches, and one language quietly renders a raw integer where the others render a formatted one. The same goes for dates: a date argument that lost its type renders an ISO timestamp at the reader.

Do not compare argument styles. Currency style against plain number style is a presentation choice that legitimately differs per language, and a guard that flags it teaches people to ignore the guard. The type is structural, the style is taste, and only one of those belongs in a test.

Compare branch names, not branch text. A plural that the translator flattened keeps its argument, so a placeholder comparison passes it happily, while the screen ends up showing a bare number with no noun after it. We had exactly that in our Dutch catalogue, and that is how it was found.

Plurals are not a solved problem you can copy

The instinct is to require that a translation has the same plural branches as English. English has two: one and other. That requirement is correct for the languages that also have two, and it is precisely wrong for the ones that do not, because it happily accepts a catalogue that is missing categories its own language needs.

Some measurements, all of them from the standard rules that ship with any modern runtime:

The Romanian row has a consequence that shows up as raw volume. Romanian puts a preposition before the noun above nineteen, so a phrase that needs no plural at all in English needs one in Romanian. Counting the messages in our catalogues that contain a plural: English has 319, Romanian has 498. 179 messages need a plural construction in Romanian that English does not need at all. Ukrainian adds 13 of its own, French 4.

Here is one of them, and it is completely ordinary English:

en: Start {days}-day trial
uk: Почати пробний період на {days, plural, one {# день} few {# дні} many {# днів} other {# дня}}

Nothing about the English sentence hints that it is a plural problem. A guard that derives its expectations from English cannot possibly see this.

So the second guard runs per language against its own catalogue: for every plural in it, is every category that this language can actually produce handled, either by name or by an exact-value branch?

That guard needs one deliberate limit, or it becomes useless. French and Spanish have a many category that applies only to exact multiples of a million. Requiring it everywhere would add hundreds of branches no human will ever see, in a product that counts servers, addresses, team members and days. So the check is sample driven: it asks which categories are reachable for the counts a key realistically sees, and there is a small explicit list of keys that count higher, so the machinery is there on the day something does.

That list is currently empty, and that is a measurement rather than an oversight. The two candidates that look like they might grow large count bytes, and the French rule wants an exact multiple of a million; a file of precisely 1,000,000 bytes is a coincidence, not a case.

The guard that was silently doing nothing

Now the uncomfortable one, because it is the failure this whole article is about, happening to the tooling itself.

We have a check that every translation call in the code refers to a key that exists. It parses the source, tracks which translator object belongs to which namespace, and looks each key up. It reported that it covered several thousand keys. It was green.

It was also skipping every call that came from a destructuring pattern, because the code that bound a translator to a namespace only handled a plain identifier and silently ignored anything else. No binding, therefore no key to check, therefore no failure. The pattern it skipped happened to be the one used throughout most of our application.

We found it because somebody guessed a key name, wrote it, and the test stayed green.

Two rules came out of that, and we now apply both to every static analysis we write.

An analyser that meets a shape it does not understand must fail loudly, not skip quietly. Unrecognised is a finding, not a no-op. The whole value of the tool is that it looked at everything.

Prove the guard red before you trust it green. Break something on purpose and watch the test fail. A check that has never failed has not been tested; it has only been run.

A close relative of that lesson: a brand name in our catalogues was spelled four different ways, and the one place in the whole codebase that knew the correct spelling knew it as an exception in an allowlist. The only component that had the right answer was configured to look away from it. An exception is not a guard, and the fix is usually to invert the thing: state the rule, and let the exceptions be data hanging off it.

Parsing your own message format is harder than it looks

Placeholder extraction with a regular expression works until it does not, and the way it fails is instructive because it fails on your best strings.

A select branch whose text is a single word looks exactly like an argument to a pattern matcher. Our old scanner raised false alarms on the most carefully written messages in the catalogue, and at the same time missed real arguments inside branches, because a regular expression cannot count braces. It was wrong in both directions at once.

The replacement walks braces properly and recurses into branch bodies. It is thirty lines instead of one, and it is the difference between a guard people trust and a guard people annotate their way around.

Typography is a per-language guard, not a shared one

The last category cannot be checked by comparison at all, because it is exactly the thing that differs between languages. French wants a non-breaking space before a colon, a semicolon, an exclamation mark and a question mark. Ukrainian wants a specific apostrophe character that looks like the two other apostrophes you probably have on your keyboard. Spanish opens a question with an inverted mark, at the point where the question starts rather than at the start of the sentence. Romanian uses its own quotation marks and comma-below diacritics.

None of that is comparable across languages, so each language that has rules gets its own check. That sounds like duplication and is not: the alternative is a single generic guard that can only assert the intersection of nine sets of rules, which for practical purposes is nothing.

One warning from doing this in a large repository: a non-breaking space is invisible, and several tools will helpfully convert it to a normal space on the way through. We put them in with a script for exactly that reason, and check afterwards that they are still there.

What we would build first, in order

If you are starting from a single language and expecting to add more, this is the order that would have saved us the most.

1. 01Same keys in every catalogue, checked in CI. Cheap, and it is the foundation the rest stands on.
2. 02Placeholder and argument-type comparison against the reference language. This is the one that catches the silent bugs.
3. 03Branch-name comparison for plurals and selects, so a flattened plural is a failure.
4. 04Per-language plural coverage, driven by the language's own rules, before the first language that needs more categories than your reference arrives.
5. 05Per-language typography, one small check per language that has conventions.
6. 06A static check that every key referenced in code exists and, more importantly, that it is honest about the calls it could not understand.

The theme running through all six is that translation bugs do not throw. They render. Every guard here exists because something rendered perfectly, in a language nobody in the room reads, and said something slightly different from what we meant.