Transformers are inherently succinct
An ICLR 2026 paper proves fixed-precision transformers can describe some formal languages with extreme compactness.
This paper studies transformer expressivity through succinctness: how compactly a formalism describes a language. It proves fixed-precision transformers can be exponentially more succinct than LTL and RNNs, and doubly exponentially more succinct than finite automata. The same succinctness makes verification hard, with basic problems such as emptiness and equivalence shown to be EXPSPACE-complete.
This ICLR 2026 paper, "Transformers are inherently succinct," revisits the expressive power of transformers from a theoretical perspective. Much prior work has asked: under fixed precision, which formal languages can transformers, RNNs, finite automata, or linear temporal logic (LTL) each recognize? The authors argue that asking only "can it express this" is not enough, because two formal systems may both be able to express the same class of languages yet require descriptions of completely different sizes. They therefore introduce the concept of "succinctness" from logic and automata theory, focusing on how large the minimal representation of a given language must be when described by different systems.
Free shows the 3-line summary; Pro unlocks the full deep summary (~300 words) so you never have to click through.
See Pro plans →Want the original English / full article?
Read on Hacker News (AI keywords) →Summaries are AI-generated; the original article is authoritative.