187 Languages on Paper: An Audit of a Machine-Translated Multilingual Sensitive-Word Lexicon

Abstract

A keyword list is the cheapest fallback for content moderation when a classifier is unavailable, and a machine-translated keyword list is the cheapest way to cover many languages. This paper audits one such artifact: a public lexicon that advertises 187 languages, five categories, and 2,951,608 entries. Every advertised language directory exists, every one of the five category files is present, and the JSON parses without error. That is where the simple story ends. Of the 2,951,608 entries, 2,685,296 are copies of the same 14,594-term URL list, passed through unchanged in every language, and 218,960 are semantic translations of 15,784 Chinese source terms in the 184 non-Chinese targets; the remainder belongs to the three Chinese variants. Only 70.9% of the semantic target entries differ from their source. The non-Chinese targets split into two sharply separated groups: 113 languages with 0.71% identity on Han-containing source entries, and 71 languages with 72.2%. The partial group is not a set of 71 different failures. Its members leave almost exactly the same source entries untranslated---the pairwise Jaccard similarity to the modal failure set is 99.98% for political terms and 99.62% for porn terms---and only 18 distinct failure signatures appear across the 71 languages. The pattern is consistent with a pipeline that stopped on specific inputs, not with 71 languages that are individually hard to translate. An expected-script check separates real script coverage from superficially non-empty output: 13 languages exceed 98.5%, while Japanese kana reach 64.6% (an all-kanji lower bound) and Serbian reaches 27.9%. A duplicate analysis finds 5,933 excess duplicate occurrences, and 26,681 translated entries (17.2%) are shared verbatim across at least two languages, rising to 46.2% for political terms. A 40-pair inspection sample of benign advertising terms finds 27 accurate, 7 acceptable, 4 wrong-sense, and 2 other outcomes. The paper does not print the sensitive source strings; it reports the audit method, the coverage arithmetic, and what a maintainer should fix first.

Body

Imported entry paper (unverified hypothesis)

This paper was imported into jetessays as an entry paper from the author's local archive (sensitive-words-audit). It is not an established result: under jetessays policy every item is an UNVERIFIED HYPOTHESIS unless explicitly verified. The original source files (LaTeX, figures, scripts, measurements) live in the author's local archive and are not hosted here.

Abstract (as submitted)

A keyword list is the cheapest fallback for content moderation when a classifier is unavailable, and a machine-translated keyword list is the cheapest way to cover many languages. This paper audits one such artifact: a public lexicon that advertises 187 languages, five categories, and 2,951,608 entries. Every advertised language directory exists, every one of the five category files is present, and the JSON parses without error. That is where the simple story ends. Of the 2,951,608 entries, 2,685,296 are copies of the same 14,594-term URL list, passed through unchanged in every language, and 218,960 are semantic translations of 15,784 Chinese source terms in the 184 non-Chinese targets; the remainder belongs to the three Chinese variants. Only 70.9% of the semantic target entries differ from their source. The non-Chinese targets split into two sharply separated groups: 113 languages with 0.71% identity on Han-containing source entries, and 71 languages with 72.2%. The partial group is not a set of 71 different failures. Its members leave almost exactly the same source entries untranslated---the pairwise Jaccard similarity to the modal failure set is 99.98% for political terms and 99.62% for porn terms---and only 18 distinct failure signatures appear across the 71 languages. The pattern is consistent with a pipeline that stopped on specific inputs, not with 71 languages that are individually hard to translate. An expected-script check separates real script coverage from superficially non-empty output: 13 languages exceed 98.5%, while Japanese kana reach 64.6% (an all-kanji lower bound) and Serbian reaches 27.9%. A duplicate analysis finds 5,933 excess duplicate occurrences, and 26,681 translated entries (17.2%) are shared verbatim across at least two languages, rising to 46.2% for political terms. A 40-pair inspection sample of benign advertising terms finds 27 accurate, 7 acceptable, 4 wrong-sense, and 2 other outcomes. The paper does not print the sensitive source strings; it reports the audit method, the coverage arithmetic, and what a maintainer should fix first.

Note

Imported via the jetessays seed importer. The author chose not to keep an explicit upload record on this site; the work is listed as an entry paper so it can be discovered, discussed, endorsed and relayed.

Discussion

Log in to join the discussion

← back to browse