讓 Token 持續流動：來自 16 個開源強化學習（RL）函式庫的啟示★ 85
Hugging Face Blog96 days agoCommentaryWith the success of reasoning models such as DeepSeek-R1, reinforcement learning (RL/RLHF) has become a critical technique for improving the alignment and…
Ulysses 序列平行化：實現百萬 Token 超長上下文的模型訓練技術解析★ 78
Hugging Face Blog97 days agoTutorialAs large language models (LLMs) push the demand for long context toward the million-token scale, the VRAM of a single GPU can no longer accommodate the…
釋放 GPT 開源模型的 Agentic RL 訓練潛力：LinkedIn 實務回顧與反思★ 75
Hugging Face Blog138 days agoCommentaryThis article, published on the Hugging Face blog and authored by the LinkedIn team, is a practical retrospective whose core subject is how to unlock "Agentic…
Hugging Face 推出全新資料集串流技術：效率提升 100 倍★ 85
Hugging Face Blog230 days agoReleaseHugging Face's official blog recently published a major update announcing a comprehensive overhaul of the streaming mode in its core open-source library…
Nemotron-Personas-India：為主權 AI 打造的印度在地化合成數據集★ 75
Hugging Face Blog243 days agoReleaseAs "Sovereign AI" becomes a global trend, countries around the world are actively seeking to build AI models that reflect their own culture, values, and…
Hugging Face Accelerate ND-Parallel 指南：高效多 GPU 訓練完全解析★ 80
Hugging Face Blog310 days agoTutorialAs the parameter counts of generative AI and large language models (LLMs) push into the tens and hundreds of billions, the memory of a single GPU has long been…
Mini-R1：重現 DeepSeek-R1「頓悟時刻」的 RL 強化學習教學★ 85
Hugging Face Blog499 days agoTutorial### Background and the Mystery of the "Aha Moment" Following the release of DeepSeek-R1, a wave of excitement around "Reasoning Models" swept the AI community…
修正梯度累積：解決 LLM 微調中常被忽視的數學偏差★ 85
Hugging Face Blog606 days agoTutorial### The Mathematical Flaw in Traditional Gradient Accumulation Gradient accumulation is an extremely common technique in deep learning. When VRAM is limited…
透過 Flash Attention 2 的 Packing 技術提升 Hugging Face 訓練效率★ 80
Hugging Face Blog662 days agoTutorialWhen fine-tuning or pre-training large language models (LLMs), the sequence lengths of input data are typically uneven. The traditional approach is to use…
GaLore：在消費級硬體上訓練大型語言模型的突破性技術★ 85
Hugging Face Blog816 days agoReleaseAs the parameter counts of large language models (LLMs) have skyrocketed, the hardware requirements for training and fine-tuning these models have risen…
深入剖析：使用 PPO 進行 RLHF 的 N 個關鍵實作細節★ 85
Hugging Face Blog964 days agoTutorialThis technical blog post from Hugging Face takes an in-depth look at the critical "implementation details" that are routinely glossed over in academic papers…
Hugging Face 案例研究：Writer 如何利用 Hugging Face 構建複雜的企業級生成式 AI 應用
Hugging Face Blog1,079 days agoBusinessThis case study takes an in-depth look at how Writer, an enterprise-grade generative AI platform, leverages the Hugging Face open-source ecosystem and…
BigCode 背後的大規模近乎重複資料刪除技術★ 75
Hugging Face Blog1,125 days agoTutorialThis technical blog post from Hugging Face takes an in-depth look at the challenges the BigCode project (the collaborative initiative behind StarCoder) faced…
Databricks 與 Hugging Face 深度合作：大型語言模型（LLM）訓練與微調速度提升高達 40%★ 70
Hugging Face Blog1,145 days agoBusinessThis case study introduces a deep technical collaboration between Databricks and Hugging Face, aimed at addressing the efficiency and cost challenges…
圖解人類回饋強化學習 (RLHF)：ChatGPT 背後的關鍵對齊技術★ 85
Hugging Face Blog1,283 days agoTutorialThe release of ChatGPT in late 2022 triggered an explosion in generative AI, and the most critical technology behind it is Reinforcement Learning from Human…
如何使用 Megatron-LM 訓練大型語言模型：Hugging Face 實戰指南★ 72
Hugging Face Blog1,376 days agoTutorialAs language model scales continue to expand, the memory (VRAM) of a single GPU has long been unable to accommodate models with tens or hundreds of billions of…
使用 DeepSpeed 與 Hugging Face Accelerate 加速超大型模型訓練★ 75
Hugging Face Blog1,447 days agoTutorialThis official Hugging Face blog post provides a detailed walkthrough of how to combine the `Accelerate` library with Microsoft's `DeepSpeed` deep learning…
透過 DeepSpeed 與 FairScale 的 ZeRO 技術，讓 Hugging Face 訓練容納更多參數且速度更快★ 80
Hugging Face Blog1,972 days agoReleaseAs the parameter scale of Transformer models (such as GPT, T5, etc.) grows exponentially, deep learning faces a severe "Memory Wall" challenge. With limited…