一個失敗的實驗：Infini-Attention，以及為什麼我們應該繼續嘗試？
Original: A failed experiment: Infini-Attention, and why we should keep trying?
This Hugging Face blog post provides a detailed account of the team's attempt to reproduce and evaluate Google's proposed…
Google 提出的 Infini-Attention 曾承諾能實現無限長度上下文，但 Hugging Face 團隊在實際重現與測試後發現效果不如預期。實驗顯示，該技術採用的「壓縮記憶體」機制存在嚴重的資訊損失，在精確檢索任務（如大海撈針）中表現不佳，且訓練過程極不穩定。儘管這是一次失敗的嘗試，但團隊強調分享「負面結果」對於 AI 社群避免重蹈覆轍、探索更有效的長上下文解決方案至關重要。
This Hugging Face blog post provides a detailed account of the team's attempt to reproduce and evaluate Google's proposed "Infini-Attention" mechanism — and their ultimate conclusion that it fell short in practice. Infini-Attention was originally introduced in Google's paper "Leave No Context Behind," which claimed that by combining "compressive memory" with "local masked attention," Transformer models could handle infinitely long contexts within a fixed memory budget.
Free shows the 3-line summary; Pro unlocks the full deep summary (~300 words) so you never have to click through.
See Pro plans →Want the original English / full article?
Read on Hugging Face Blog →Summaries are AI-generated; the original article is authoritative.