Clustering 3x Jetson Nano Orin Supers for Distributed AI
r/LocalLLaMA top day7 days agoTutorialA developer has shared a practical guide on clustering three NVIDIA Jetson Nano Orin Super boards, leveraging their Ampere CUDA cores and unified memory. This project is part of 'smolcluster,' an initiative to make distributed AI training and inference accessible using everyday hardware like Macs, Raspberry Pis, and Jetsons. The series aims to explore whether heterogeneous clusters (mixing different hardware architectures) can effectively run local LLMs.
使用 Hugging Face 與 Dask 實現大規模 AI 資料處理★ 72
Hugging Face Blog613 days agoTutorialAs the scale of AI models and the volume of training data grow dramatically, the computational capacity and memory (RAM) of a single machine often become…
優化故事：BLOOM 超大模型推理優化實踐
Hugging Face Blog1,341 days agoTutorialThis technical blog post from Hugging Face documents in detail the practical process of optimizing inference for BLOOM, the open-source multilingual large…
使用 DeepSpeed 與 Accelerate 實現極速 BLOOM 模型推理
Hugging Face Blog1,367 days agoTutorialBLOOM is a massive open-source multilingual model with 176 billion parameters. Running BLOOM at FP16 precision requires at least 352 GB of video memory (VRAM)…
使用 Hugging Face Transformers 與 Ray 實現大規模檢索增強生成 (RAG)
Hugging Face Blog1,950 days agoTutorialRetrieval-Augmented Generation (RAG) is a powerful architecture that combines a "retriever" with a "generator." It enables language models to dynamically…