Tiny hackable CUDA language model implementation
Hacker News (AI keywords)9 days agoNew ToolThis GitHub project implements a compact generative pretrained transformer as an autoregressive byte-level sequence model. Its README describes causal self-attention, RoPE, feed-forward layers, AdamW, cross-entropy training, and BLAS/OpenBLAS-backed matrix operations, with CUDA toolkit listed in setup steps. It is most useful as an educational and experimental codebase, not as a production-grade replacement for large commercial LLMs.
從零開始在 nanoVLM 中實作 KV Cache★ 75
Hugging Face Blog375 days agoTutorialIn the inference process of large language models (LLMs) and vision-language models (VLMs), autoregressive decoding is a major performance bottleneck. Each…
你也能設計出最先進的 Transformer 位置編碼：從直覺到 RoPE 的數學推導★ 75
Hugging Face Blog566 days agoTutorialThis educational article from Hugging Face aims to guide readers — in the most intuitive, step-by-step way — to "reinvent" RoPE (Rotary Position Embedding)…
萬事通，局部精通：Hugging Face 發表多功能 Transformer 代理人 JAT★ 75
Hugging Face Blog783 days agoReleaseIn the field of artificial intelligence, developing a "Generalist Agent" — one capable of chatting, writing, controlling robots, and playing video games all at…
Hugging Face 整合 PatchTST：專為時間序列預測設計的 Transformer 模型★ 75
Hugging Face Blog864 days agoReleaseThe official Hugging Face blog announced a major update: the integration of the PatchTST (Patch Time Series Transformer) model into its `transformers`…
介紹 RWKV：兼具 Transformer 優勢的全新 RNN 架構★ 75
Hugging Face Blog1,126 days agoReleaseHugging Face has announced official support for RWKV (Receptive Weighted Key Value) models in its `transformers` library. RWKV is an innovative architecture…
Nyströmformer：透過 Nyström 方法以線性時間與記憶體複雜度逼近 Self-Attention
Hugging Face Blog1,412 days agoReleaseThis Hugging Face blog post provides a detailed introduction to Nyströmformer, a Transformer variant designed to overcome the bottleneck of processing long…
Hugging Face 與 Habana Labs 合作加速 Transformer 模型訓練
Hugging Face Blog1,524 days agoReleaseHugging Face and Intel's Habana Labs have officially announced a partnership aimed at providing the community with more efficient and cost-effective solutions…
BERT 101：最先進的 NLP 模型完整原理解析
Hugging Face Blog1,565 days agoTutorialBERT (Bidirectional Encoder Representations from Transformers) is a landmark natural language processing (NLP) model proposed by Google in 2018. This Hugging…
Perceiver IO：可擴展且適用於任何模態的全注意力機制模型★ 70
Hugging Face Blog1,642 days agoReleaseThis article introduces DeepMind's Perceiver IO model and its integration into the Hugging Face Transformers library. Traditional Transformer models, while…
使用區塊稀疏矩陣（Block Sparse Matrices）打造更小、更快的語言模型
Hugging Face Blog2,103 days agoTutorialIn the field of natural language processing (NLP), the Transformer architecture has become the dominant paradigm, but its core self-attention mechanism…
Reformer：挑戰語言模型長文本處理極限的架構
Hugging Face Blog2,172 days agoPaperThis technical blog post published by Hugging Face takes a deep dive into how the Reformer architecture overcomes the memory and computational bottlenecks that…