A Transformer That Learned Only the Marginal: A Reproducibility Audit of a Published 19.2M-Parameter Edge Music Model

Abstract

MIPIIL is a published model presented as an edge-oriented music generator: a 19,197,204-parameter Transformer over a byte-level BPE alphabet, with a model card that promises unlimited-length real-time generation from a 375,000-file MIDI corpus. This paper reports an audit of that artefact, and the audit is a negative result about sequence modelling rather than about audio quality. The checkpoint loads exactly as documented and its weights are finite and meaningfully trained, yet the next-token distribution it defines does not depend on its input: over contexts of 16 to 512 tokens taken from a real Standard MIDI File, the mean pairwise total variation between next-token laws is below 1.4×10⁻⁷, the number of distinct argmax positions is exactly one, and the mutual information between context and prediction is 0.000 nats. An untrained control of the same architecture is measurably more context sensitive (2.12×10⁻² versus 9.08×10⁻⁸ total variation at the logits). Forward hooks localise the collapse: the feed-forward branch norm grows to 541.6 while the LayerNormed residual stream stays near √d, and cross-context spread contracts by a factor of 209,963 with depth in the trained model against 2.0 in the control. The consequence is that the artefact is, to within 5×10⁻⁶ bits per token, a 276-entry histogram that costs 5.9613 bits, and it codes real MIDI at 5.4461–5.4541 bits per token while a held-out unigram of the same file needs 4.454. We also document the release mechanics that let this ship: greedy decoding repeats one token forever, none of 40 sampled generations contains a MIDI header, the published decoder raises on the checkpoint's own special tokens, and the shipped generator never decodes a token at all — it maps token ids to pitches by modular arithmetic. Because the model audited here is the author's own release, the paper is offered as a verification protocol rather than as a critique: three measurements, each under a minute of compute, would have caught all of it.

Body

Imported entry paper (unverified hypothesis)

This paper was imported into jetessays as an entry paper from the author's local archive (mipiil). It is not an established result: under jetessays policy every item is an UNVERIFIED HYPOTHESIS unless explicitly verified. The original source files (LaTeX, figures, scripts, measurements) live in the author's local archive and are not hosted here.

Abstract (as submitted)

MIPIIL is a published model presented as an edge-oriented music generator: a 19,197,204-parameter Transformer over a byte-level BPE alphabet, with a model card that promises unlimited-length real-time generation from a 375,000-file MIDI corpus. This paper reports an audit of that artefact, and the audit is a negative result about sequence modelling rather than about audio quality. The checkpoint loads exactly as documented and its weights are finite and meaningfully trained, yet the next-token distribution it defines does not depend on its input: over contexts of 16 to 512 tokens taken from a real Standard MIDI File, the mean pairwise total variation between next-token laws is below 1.4×10⁻⁷, the number of distinct argmax positions is exactly one, and the mutual information between context and prediction is 0.000 nats. An untrained control of the same architecture is measurably more context sensitive (2.12×10⁻² versus 9.08×10⁻⁸ total variation at the logits). Forward hooks localise the collapse: the feed-forward branch norm grows to 541.6 while the LayerNormed residual stream stays near √d, and cross-context spread contracts by a factor of 209,963 with depth in the trained model against 2.0 in the control. The consequence is that the artefact is, to within 5×10⁻⁶ bits per token, a 276-entry histogram that costs 5.9613 bits, and it codes real MIDI at 5.4461–5.4541 bits per token while a held-out unigram of the same file needs 4.454. We also document the release mechanics that let this ship: greedy decoding repeats one token forever, none of 40 sampled generations contains a MIDI header, the published decoder raises on the checkpoint's own special tokens, and the shipped generator never decodes a token at all — it maps token ids to pitches by modular arithmetic. Because the model audited here is the author's own release, the paper is offered as a verification protocol rather than as a critique: three measurements, each under a minute of compute, would have caught all of it.

Note

Imported via the jetessays seed importer. The author chose not to keep an explicit upload record on this site; the work is listed as an entry paper so it can be discovered, discussed, endorsed and relayed.

Discussion

Log in to join the discussion

← back to browse