🔥 Trending today
PSA: Throttle GPU Power Limits for Major Energy Savings with Minimal Inference Performance Loss
r/LocalLLaMA top day5 days agoHardwareA Reddit user reminds the local LLM community that throttling GPU power limits offers outsized energy savings with minimal performance cost. On dual Radeon VII cards, cutting power from 250W to 100W per card resulted in less than 10% drop in inference speed. LLM inference is memory-bound rather than compute-bound, making it uniquely tolerant of reduced GPU clock speeds compared to training or rendering tasks.
在地端構建 AI：Hugging Face 聯手戴爾推出 Dell Enterprise Hub★ 75
Hugging Face Blog754 days agoReleaseHugging Face and Dell Technologies have announced the launch of the "Dell Enterprise Hub," a new solution designed for enterprise on-premise AI deployment. As…
Hugging Face 揭秘：🤗 Accelerate 如何藉助 PyTorch 運行超大型模型★ 80
Hugging Face Blog1,356 days agoTutorialAs the parameter counts of large language models (LLMs) grow exponentially, how to load and run these models on limited hardware has become a major pain point…