iCreatAI News
Back to main site
Skip to content

Technology areas

On-device AI Latest updates

AI running on phones, computers, and edge devices: progress in small models, local inference, and on-device chips.

8 条精选Past 30 days: 7 updatesTotal: 85 updates

Updated

On-device AI的精选

Nov 13Fri 1–8 updates
  1. Apple Newsroom74

    Apple HomeView available today

    Apple announced that HomeView, a device that recognizes up to six people by face and shows their personal content in Messages, FaceTime, Calendar, Reminders, and Find My, is available to order today in Australia, Canada, India, New Zealand, Singapore, the UK, and the U.S. for $299 (U.S.), with a wall-mounted version also at $299. It runs Siri AI, which offers onscreen awareness and, when paired with an Apple Intelligence-enabled iPhone or iPad, personal context and cross-app actions; Siri AI is available in beta in HomeView software version 27.2. HomeView is not for sale in the European Union because Siri AI is currently unavailable in iOS in the EU, and Schneider Home mounts start at $59.95 (U.S.).

    Why it matters: The announcement details pricing, availability, and Siri AI features for a new Apple device, useful for readers tracking Apple's home products and regional AI rollout.

  2. Hacker News AI88

    Edge0-145B: Frontier AI Capabilities on Your Own Computer

    The company open-sourced Edge0-145B and Edge0-120B, describing Edge0-145B as the first open-source 145-billion-parameter on-device model, alongside a unified inference framework and deployment tools. It reports 29GB peak memory, 80+ tokens/s decoding, 1,100+ tokens/s prefill, roughly 120W typical power and 128K–256K context on a compatible laptop, with benchmark scores compared against named cloud models. The release also describes shared-matrix sparse experts, tiered GPU/system-memory/SSD loading and CPU-GPU co-scheduling, plus local demonstrations in video, Blender 3D scenes, games, drone control and PCB design.

    Why it matters: It offers a detailed, benchmark-backed look at how a 100B+ model is engineered to run locally, useful for readers weighing on-device versus cloud AI.

Nov 11Wed
  1. PyTorch Blog71

    ExecuTorch WebGPU: PyTorch Models on the GPU, in the Browser

    ExecuTorch's new WebGPU backend runs PyTorch models exported with torch.export to a .pte file directly on the GPU in WebGPU-enabled browsers, without converting to another framework's format. The same artifact runs across Chrome and Safari on Metal, Vulkan and D3D12, and on an Apple M1 Pro in Chrome it was 4–7× faster than ExecuTorch's Wasm (XNNPACK) CPU path on prefill and 2–10× faster on decode. Validated models include Llama 3.2, Qwen3, Gemma 3, SmolVLM, Depth Anything V2, SAM 2, Whisper and on-device LoRA fine-tuning, all running locally in the tab.

    Why it matters: It documents a concrete path for running PyTorch models on browser GPUs with benchmark numbers and a list of validated models, useful for developers weighing in-browser inference.

  2. Google Developers Blog72

    On-Device Speech Recognition and Text-to-Speech in Google AI Edge Gallery

    Google introduced on-device ASR and TTS in Google AI Edge Gallery, pairing lightweight speech models with on-device SLMs like Gemini Nano and Gemma 4 for fully offline voice interaction. ML Kit on Android offers multilingual transcription across 100+ languages at 10x real-time speed plus a few-step diffusion TTS that starts in ~100 ms, while MediaPipe Tasks and LiteRT bring cross-platform support for open-weight models including Parakeet-TDT 0.6B, PocketTTS 100M, Kokoro-82M, and Phonon 31M. The release powers the XR Language Teacher showcase and voice-enabled AI Chat, with open-source reference code on GitHub.

    Why it matters: It documents concrete on-device speech model releases, latency and accuracy figures, and open-source code that developers can use to build offline voice pipelines.

Nov 6Fri
  1. Artificial Analysis Articles62

    Nanbeige has released Nanbeige4.3-33B scoring 20 on the Intelligence Index

    Nanbeige, the foundation model research team at BOSS Zhipin (operated by Kanzhun Limited), has released Nanbeige4.3-33B, an Apache 2.0 open-weights dense reasoning model with text-only input and output and a 524K-token context window. It scores 20 on the Artificial Analysis Intelligence Index v5, matching Qwen3.8 27B (xhigh) at 20, with relative strengths in terminal use (30% on Terminal-Bench 4.0 vs. 6%) and physics reasoning (16% on CritPt v2's Hard Subset vs. 4%), but trails on AA-Briefcase v1.1 (1162 vs. 1391 Elo), FrontierCode 2.0 (11% vs. 15%) and HLE-Diamond (7% vs. 15%). It shows high abstention and low hallucination on Omniscience (18% vs. 29% hallucination rate, declining 71% of questions) and used 158K output tokens per Intelligence Index task vs. 110K for Qwen3.8 27B.

    Why it matters: Useful for readers tracking small open-weights models, since it sets a new Pareto point for Intelligence Index versus total parameters and details trade-offs against Qwen3.8 27B.

Nov 5Thu
  1. Hugging Face Blog71

    A Closer Look at Phonon TTS. Launching High-Quality On-Device Speech That Tops the Benchmarks.

    Gradium details Phonon, a small on-device text-to-speech model released two weeks ago that needs no server or network, with Phonon 31m (English) under Apache 2.0 and Phonon 40m covering English, French, German, Spanish and Portuguese under a non-commercial license. The models, about 65 MB and 75 MB, offer over 100 built-in voices plus voice cloning and voice design, and on Hugging Face's Open TTS Leaderboard Phonon ranks first in German and Spanish on intelligibility and tops the CPU streaming benchmark on time-to-first audio. The post also covers the ptts inference framework, Transformers integration from v5.21, launch demos, and more than 200,000 downloads in the first two weeks.

    Why it matters: It offers a detailed look at how a small on-device TTS model was built, benchmarked and integrated into developer tooling.

Oct 28Wed
  1. Hugging Face Blog78

    Build specialized multimodal models with LFM3

    Liquid AI introduces LFM3, a generation of Liquid Foundation Models in the 2B parameter range that places text, vision, audio, omni, and decision models on one shared backbone built from LFM3-1.7B-Base, which was pre-trained on 22T tokens and uses a tri-hybrid layer mix with a 256K context window. The release includes six 2B-class models, new in-house vision and audio encoders, and LiquidCodec, with every model fine-tunable on a single GPU and the text model fitting in about 1 GB at 4-bit precision. All LFM3 models are open-weight and available on Hugging Face, with fine-tuning notebooks and agent harness guides provided.

    Why it matters: The post is a detailed technical release covering architecture changes, benchmark comparisons, and runnable code for six models, useful for developers evaluating small multimodal models for on-device deployment.

Oct 7Wed
  1. Google DeepMind72

    EmbeddingGemma 2: an open, lightweight multimodal embedding model

    Google DeepMind announced EmbeddingGemma 2, a 740-million-parameter model built on the Gemma 4 architecture and released under the Apache 2.0 license, which maps text, code, images, video, and audio into a unified embedding space for on-device use. It offers modular encoders (as little as 270M parameters for text-only, with optional 170M vision and 300M audio encoders), Matryoshka Representation Learning to truncate vectors from 768 down to 128 dimensions, an 8K token context window, and on a Google Pixel 11 Pro requires as little as ~191MB active RAM for text-only weights and ~567MB for the full multimodal model. The company reports a 9.92-point improvement on MTEB Code (68.76 to 78.68) over EmbeddingGemma and says the model is available on Hugging Face and Kaggle, with support across tools including transformers, vLLM, llama.cpp, Ollama, and Qdrant.

    Why it matters: It documents a new open, commercially licensed embedding model with concrete size, memory, and benchmark figures relevant to developers building on-device search and RAG.