A Published "Fourier Language Model" Is a Retrieval Index, Not a Model: Provable Order Blindness and a Held-Out Audit of a 134 MB Chat Artefact

Abstract

A public model repository named F_Structure_LM advertises "a Fourier transform that replaces the self-attention of a Transformer" and shows a fluent reply to a prompt that is not in its training data. We read all 140,569,669 bytes of its payload and report what it is: 1,849,576 rows of 17 float32 numbers, each paired with one token label, plus three integers — and no learnable parameter of any kind. Inference is cosine nearest-neighbour lookup in that table. Because each row is the magnitude spectrum of raw byte-level BPE identifiers, two failure modes are theorems rather than measurements: the spectrum is invariant to time reversal, so reversing a context cannot change what is retrieved (99.05% of 4,000 held-out contexts retrieve the identical neighbour), and the zeroth bin is the unordered sum of the identifiers, carrying 24.0% of a typical key's energy. On the official LCCC valid and test splits the published index predicts the next token correctly for 2.90% of 4,000 sampled contexts — below its own constant-label baseline (5.48%) and far below a unigram over the same corpus (34.48%), against bigram 18.05%, trigram 11.63%, exact 12-gram 0.43%. Rebuilding the identical feature recipe on 50,000 sampled training dialogues, with 92.7 rows per dialogue instead of the published 37.0, lifts the same decoder to 21.63%: the artefact's failure sits in index density and label coverage, not in order blindness alone. At generation time it lands on points of its own table (mean top-1 cosine 0.998–1.000 over 896 steps), none of the three replies printed on its model card reproduce, and 32.4% of rows carry a label that cannot be decoded without an adjacent one. What it does genuinely well is cheap: 139 tokens/s and 841 MB resident on a CPU, no GPU. We close by stating what a "Fourier language model" claim would have to measure to be checkable.

Body

Imported entry paper (unverified hypothesis)

This paper was imported into jetessays as an entry paper from the author's local archive (fourier-lm). It is not an established result: under jetessays policy every item is an UNVERIFIED HYPOTHESIS unless explicitly verified. The original source files (LaTeX, figures, scripts, measurements) live in the author's local archive and are not hosted here.

Abstract (as submitted)

A public model repository named F_Structure_LM advertises "a Fourier transform that replaces the self-attention of a Transformer" and shows a fluent reply to a prompt that is not in its training data. We read all 140,569,669 bytes of its payload and report what it is: 1,849,576 rows of 17 float32 numbers, each paired with one token label, plus three integers — and no learnable parameter of any kind. Inference is cosine nearest-neighbour lookup in that table. Because each row is the magnitude spectrum of raw byte-level BPE identifiers, two failure modes are theorems rather than measurements: the spectrum is invariant to time reversal, so reversing a context cannot change what is retrieved (99.05% of 4,000 held-out contexts retrieve the identical neighbour), and the zeroth bin is the unordered sum of the identifiers, carrying 24.0% of a typical key's energy. On the official LCCC valid and test splits the published index predicts the next token correctly for 2.90% of 4,000 sampled contexts — below its own constant-label baseline (5.48%) and far below a unigram over the same corpus (34.48%), against bigram 18.05%, trigram 11.63%, exact 12-gram 0.43%. Rebuilding the identical feature recipe on 50,000 sampled training dialogues, with 92.7 rows per dialogue instead of the published 37.0, lifts the same decoder to 21.63%: the artefact's failure sits in index density and label coverage, not in order blindness alone. At generation time it lands on points of its own table (mean top-1 cosine 0.998–1.000 over 896 steps), none of the three replies printed on its model card reproduce, and 32.4% of rows carry a label that cannot be decoded without an adjacent one. What it does genuinely well is cheap: 139 tokens/s and 841 MB resident on a CPU, no GPU. We close by stating what a "Fourier language model" claim would have to measure to be checkable.

Note

Imported via the jetessays seed importer. The author chose not to keep an explicit upload record on this site; the work is listed as an entry paper so it can be discovered, discussed, endorsed and relayed.

Discussion

Log in to join the discussion

← back to browse