NVFP4 Support Merged in llama.cpp: How to Use 4-bit Blackwell Quantization
r/LocalLLaMA top day7 days agoCommentaryFollowing the merge of native NVFP4 (NVIDIA FP4) support in llama.cpp, users are exploring how to leverage this format on Blackwell GPUs (such as the RTX 50-series). The discussion focuses on converting NVFP4 safetensors (like Gemma 4 QAT) to GGUF format and whether importance matrices (imatrix) are required. This enablement promises significant performance gains for local LLM execution on next-gen hardware.
start-llama: A Handy CLI Launcher for llama-server with Easy Customization
r/LocalLLaMA top day7 days agoNew ToolA developer has released 'start-llama', a command-line utility designed to simplify launching llama-server (llama.cpp). It allows users to manage sensible default configurations, support multiple server binaries, and apply per-model or command-line overrides. This tool streamlines local LLM deployment into a single, easily configurable step.
LLaMA 週報第三週：開源 AI 生態系的狂飆起點
Replicate Blog1,184 days agoCommentaryWithin just three weeks of Meta releasing the LLaMA (Large Language Model Meta AI) model, the open-source community demonstrated an astonishing pace of…