Two names for the same article

A reader found something in our translated articles that no automated check could see: a link promising one thing that landed on a page whose heading claimed another. We wrote it down as a rule and then set out to make it a guard.

The obvious guard is "the link text must be the title". That guard is wrong, and finding out why took the whole measurement.

The obvious rule is not the house style

We measured every internal article link across nine languages: 1153 of them, with no missing targets. Then we scored each link text against the title of the page it points at, as a word overlap, and looked at everything below a threshold of 0.55.

There were 418 of them. Forty nine were in English, which is the source language and therefore by definition not a translation drift. Those forty nine look like this:

> a link reading only ever connects outbound, pointing at an article titled The case against port forwarding your panel

That is not a mistake. A short descriptive link text runs inside the sentence around it and is supposed to. A guard on the low band would fire four hundred times and name zero defects, and a guard that cries wolf gets skipped, which is worse than not having one.

So the low band is house style, and the thing we were looking for is somewhere else entirely.

The defect is in the high band, and it is a replacement

The case from the report has an overlap of 0.78. It is well above the threshold, which is why looking below the threshold could never have found it.

> link: ein Ablauf ist ein Graph, und ein Graph muss endlich sein > title: Ein Flow ist ein Graph, und er muss endlich sein

The link text is not shorter than the title and it is not a paraphrase. It quotes the title and swaps a word. That is the actual shape of the defect: a word appears that the title does not contain and a word from the title is missing, at the same time.

Shortening a title is fine, because a link inside a sentence often cannot carry the whole thing. Adding a clarifying word is fine. Doing both at once means two names are in circulation for one article, and a reader who saw the link and then the heading has to decide whether they are on the right page.

Stated that way the rule is one sentence and it survives contact with real text. Above the overlap threshold, a link text may shorten the title or extend it, but not both.

Two things that look like a different name and are not

Two kinds of variation had to be filtered out before the rule was usable, and both were found by running it and reading the noise.

The first is function words: articles, conjunctions, prepositions. They are structural and they carry no name. Any word of at most two characters is dropped from both sides.

The threshold of two, rather than three, is measured and not chosen. The Italian case from the same report is deve essere against a title reading dev'essere: the same two words, written with an elision. With a limit of three characters, dev falls out of the title side, nothing is missing any more, and the finding disappears. A guard tuned one character too aggressively would have gone green on one of the two cases it was built for.

The second is inflection. sell and selling, says and saying are the same word in different forms, and in eight languages that is constant. Two words sharing a four character stem count as the same word.

The English counterpart is the triage

That left a set of findings that needed judgement, and the judgement had to come from somewhere other than taste.

The rule we used: look at the same link in the English source. If the English quotes its own title there, then the translation has drifted away from something and it is a defect. If the English is itself descriptive at that spot, the translation is following the source and it is house style.

That is a good test because it is answerable by looking, it does not require knowing the language, and it puts the decision where the source of truth already is. It produced eighteen deliberate exceptions, each recorded with the reason, which is always a sentence about what the English does at that spot rather than a note saying "checked".

The repair itself was 76 link texts across 66 files, in eight languages: fifteen in Dutch, thirteen in Italian, twelve in French, eight each in Romanian and Ukrainian, seven each in German and Spanish, six in Portuguese, and none in English. The diff came out at exactly 66 lines in and 66 lines out, which is the cheapest confirmation there is that nothing else moved. The high band count went from 94 to 18, and the 18 are the exceptions.

An allowlist needs a test that it is alive

The guard has eleven tests: one per language, one demanding that every exception carries a reason, and one demanding that every exception is actually hit.

That third one is the important one, and it comes from an expensive lesson elsewhere in this codebase. An allowlist entry with a typo in it matches nothing. It also looks complete, sits in the file for a year, and quietly protects nothing. We had a list of thirty entries in a different guard where four were dead for exactly that reason.

So the list is checked from the other side: if a line in it never fires, the test fails and the line has to go. That closes the loop, because it means an exception that has been repaired can no longer linger as decoration.

The guard was proven red before it was believed green, on the state of the tree before the repair: eight of eleven tests failed, reporting precisely the 76 findings that were then fixed.

Two things this round found and did not fix

Two findings were outside the scope and were written down instead of quietly repaired.

Five translations of one article had lost a word from the title that four other languages carried, so the same article had a different subject in different languages. That is a title problem rather than a link problem, and it was handled separately.

The Ukrainian articles spell one industry term two ways, seventeen files against six. That is a terminology decision across the whole catalogue and not a defect in seventeen files, and making it in passing while repairing something else is how a catalogue ends up with three spellings instead of two.

What we would write down

1. 01The rule you would write from the report is usually wrong. Measure the whole corpus before you encode it.
2. 02A short descriptive link text is house style, and a guard on it names hundreds of non defects.
3. 03The defect is a replacement, not a shortening. A word added and a word missing at the same time.
4. 04Look above the threshold, not below it. The real case scored 0.78 against a threshold of 0.55.
5. 05Tune limits on the cases you have, and check that the tuning does not make one of them disappear.
6. 06Inflection is not a different name. A shared stem is enough.
7. 07The source language is the triage. If English is descriptive there too, the translation is not drifting.
8. 08Every exception carries a reason, and the reason says what the source does, not that somebody looked.
9. 09Test that your allowlist is alive. A dead entry protects nothing and looks complete.
10. 10Prove the guard red on the old state, with the count of findings it should produce.

For clarity about what this is: 1153 internal article links measured across nine languages with no missing targets, 76 repaired in 66 files, a diff of exactly 66 in and 66 out, the high band reduced from 94 findings to the 18 recorded exceptions, the new guard 11 tests and proven red at 8 failures on the old tree, the content check 851 of 851 generated files byte identical, the web suite 4594 green where it was 4583, the type check clean, the linter with no errors, and a byte sweep over the 66 touched files with the two typography counts explained rather than assumed. Not proven: there is still one reader per language, none of them is a native speaker, and the exception reasons were not read by a second person.

From the platform work on Provibr. The reader review that turned this up is in three faults no script could see, and what nine catalogues do to a set of fixed controls is in sixteen thousand keys and nine languages.