Benchmarking Google Eloquent Exposes Major On-Device Dictation Reliability Issues
r/LocalLLaMA top day3 days agoBenchmarkA LocalLLaMA user tried to benchmark Google’s new fully local dictation app, Eloquent, against open ASR models such as Qwen3-ASR and NVIDIA Parakeet V3. The tester reported that roughly half of dictations returned only fragments, even during manual use. When Eloquent produced complete transcripts, its word error rate was competitive, but the missing-output behavior made the app unreliable for evaluation and practical use.
Apple Announced a New On-Device Inference Engine for Apple Silicon
r/LocalLLaMA top day5 days agoReleaseApple announced CoreAI at WWDC, which the post frames as a possible future replacement for CoreML and an alternative to MLX, llama.cpp, and torch for optimized on-device inference. Models still need conversion through Python scripts, and current supported models appear mostly from mid-2025. No performance data is available yet; the author expects it may trail MLX on GPU, but Apple’s 20B on-device foundation model claim suggests larger app-bundled models could become possible.
Apple Core AI Framework★ 76
Hacker News (AI keywords)6 days agoReleaseApple’s Core AI framework is positioned as a developer stack for deploying AI models directly inside apps on Apple silicon. The documentation describes Swift APIs, `.aimodel` assets, model specialization, caching, Xcode profiling, and debugging tools. It appears aimed at developers building low-latency, privacy-conscious on-device inference workflows, though the documentation is marked as preliminary beta information.
Gemma 4 QAT models: Optimizing model compression for mobile and laptop efficiency★ 72
Hacker News (AI keywords)9 days agoReleaseGoogle released new Gemma 4 checkpoints optimized with Quantization-Aware Training to preserve quality after compression. The release includes Q4_0 checkpoints and a mobile-focused quantization format that can reduce Gemma 4 E2B memory use to about 1GB, or below 1GB for a text-only configuration. The models are available through Hugging Face and supported across llama.cpp, Ollama, LM Studio, LiteRT-LM, Transformers.js, SGLang, vLLM, MLX, and Unsloth.
Google DeepMind 發表 Nano Banana Pro：聚焦裝置端與邊緣運算的全新輕量化模型（推測）
Google DeepMind Blog206 days agoReleaseGoogle DeepMind published a blog post on November 20, 2025 titled "Introducing Nano Banana Pro." As the full content of the original article is not publicly…
Granite 4.0 Nano：探索端側 AI 的極限，模型究竟能縮到多小？★ 75
Hugging Face Blog229 days agoReleaseThis article, jointly published by IBM and Hugging Face, delves into the technical details and application scenarios of the brand-new ultra-lightweight model…
Arm 與 ExecuTorch 0.7 聯手：將生成式 AI 推向大眾市場★ 80
Hugging Face Blog305 days agoReleaseAs generative AI advances rapidly, deploying massive models to resource-constrained edge devices — such as smartphones, smart hardware, and AI PCs — has become…
Hugging Face 發表 SmolLM3：輕量、多語言、長上下文的端側推理模型★ 80
Hugging Face Blog341 days agoReleaseHugging Face has announced the release of a brand-new generation of lightweight open-source models — SmolLM3. As the latest member of the SmolLM family…
Falcon-Edge：TII 推出一系列強大、通用且可微調的 1.58-bit 邊緣端語言模型★ 82
Hugging Face Blog395 days agoReleaseThe Technology Innovation Institute (TII) of the United Arab Emirates has officially released the "Falcon-Edge" model series on Hugging Face. This is a family…
Hugging Face 推出 SmolVLM：輕量且強大的開源視覺語言模型，可在本機高效運行★ 80
Hugging Face Blog565 days agoReleaseHugging Face has officially launched a lightweight vision language model (VLM) called **SmolVLM**, designed to bring powerful multimodal understanding…
在 Apple Silicon 上使用 Core ML 執行 Stable Diffusion★ 75
Hugging Face Blog1,291 days agoReleaseIn late 2022, Apple and Hugging Face jointly announced that Stable Diffusion had officially gained support for Apple Silicon's Core ML framework. This update…