https://llmobile.news/A concise news ticker covering AI on mobile devices, local models, apps and hardware.en-GBhttps://llmobile.news/ticker/lfm2-5-24b-a2b/https://llmobile.news/ticker/lfm2-5-24b-a2b/Tue, 13 Oct 2026 09:00:00 +0200Liquid AI has released LFM2.5-24B-A2B, a mixture-of-experts model with 24 billion parameters of which 2.45 billion are active per token. In MoE models only part of the network runs for each token, which keeps compute low while the weights stay large. The company measured 184 tokens per second on an Apple M5 Max and 114 tokens per second on an AMD Ryzen AI Max+ 395, and says the model decodes up to 1.7 times faster than comparable MoE models on-device. The context window grew from 32,768 tokens in the earlier LFM2-24B-A2B to 131,072 tokens, and the vocabulary doubled from 65K to 128K. Liquid AI scaled pre-training from 17 trillion to 33 trillion tokens. The architecture combines gated short convolutions with grouped-query attention across 40 layers and routes each token to 4 of 64 experts per MoE block. The new version is reasoning-only, so it always emits an explicit chain of reasoning before its answer. Liquid AI counts the model as leading or matching Qwen3.6-35B-A3B on five of the eight tool-use and agentic benchmarks it ran, and reports more than three times the competitor’s score on AutomationBench at 27.69 against 8.72. Qwen stays ahead on knowledge in GPQA Diamond and on coding in SWE-Bench Verified through OpenCode. The model card advises against knowledge-heavy tasks on their own and recommends pairing the model with retrieval or web search, which it says plays to its tool-calling strength. LFM2.5-24B-A2B benchmark scores Benchmark LFM2.5-24B-A2B Qwen3.6-35B-A3B AutomationBench 27.69 8.72 LiveClaw 48.09 38.19 ToolSandbox 80.71 IFEval 96.75 GPQA Diamond 74.42 83.69 SWE-Bench Verified (OpenCode) 51.00 66.20 All figures according to Liquid AI. The comparison model is Qwen3.6-35B-A3B. A demo video shows the model running inside Hermes Agent on an M5 Max MacBook Pro, planning tool calls and assembling a briefing from trending Hugging Face models without any cloud API call. Liquid AI names OpenClaw, Pi and OpenCode as further agent harnesses it trained and tested the model in. During reinforcement learning the model also learned to look up final answers in a repository’s git history or on GitHub, PyPI and Hugging Face, and according to the company it worked out that it was running in a sandbox. Your browser does not support this video. This video could not be loaded. Use the link below to open it directly. Open video Weights are on Hugging Face under the LFM License, with FP8, ONNX and MLX 4-bit variants alongside GGUF builds that run from 13.6 GB at Q4_0, a checkpoint Liquid AI trained with quantization-aware distillation. Supported runtimes include llama.cpp, MLX, vLLM, SGLang and ONNX, and the model covers 16 languages including English, German, Chinese, Japanese and Hindi. Through Liquid AI’s own API it costs $0.10 per million input tokens, $0.05 for cached input, and $1.00 per million output tokens. Source: https://www.liquid.ai/blog/lfm2-5-24b-a2b Read the article: https://llmobile.news/ticker/lfm2-5-24b-a2b/Open weightsMixture of expertsAgentsBenchmarkshttps://llmobile.news/ticker/qualcomm-amon-100b-models-phones-2028/https://llmobile.news/ticker/qualcomm-amon-100b-models-phones-2028/Thu, 08 Oct 2026 15:30:00 +0200Qualcomm chief executive Cristiano Amon says AI companies have asked his company to make phones capable of running at least a 100-billion-parameter model by 2028, with the model active almost continuously. Amon made the remarks on Alex Heath’s Sources podcast, recorded live at Qualcomm’s Snapdragon Summit in Maui, though he declined to name the companies behind the request. He said they want a model “running kind of all the time”, according to a transcription by The Next Web. Some of the companies at the forefront of AI are telling Qualcomm, ‘By 2028, I need to make sure that I have the ability to run at least a hundred billion parameter model into a phone.’ AI companies are also starting to build their own handsets, Amon said, naming ByteDance’s Doubao phone, manufactured by Nubia, as an example from China and adding that more are coming. Heath pointed out that personal agents such as Muse and Grok Bot drive browsers on virtual machines in the cloud rather than on the phone. Amon answered that some of that work will stay in data centers while some of it has to happen on the device. Humans are not going to go travel to a data center, ring the bell and say, ‘Give me some AI.’ Amon likened the moment to the early iPhone, when the applications that later defined the platform did not yet exist. He said “The smartphone was defined by the third-party applications,” and he expects agents to play that role on the next generation of phones. The launch of OpenClaw made the idea of an AI-first smartphone concrete for him, he said, describing an agent that plans, routes tasks and acts on the user’s behalf. Qualcomm’s current flagship silicon sits below that 2028 target. The company announced the Snapdragon 8 Elite Extreme Gen 6 at the same summit, and Computerbase reports that the Extreme variant gets 50 percent more dedicated NPU memory, which Qualcomm says prepares it for mixture-of-experts models, where only part of the model is active per token — the example given is a 30-billion-parameter model with roughly 3 billion active. The interview does not say how much memory or storage a 100-billion-parameter model would need on a phone. Memory prices have risen five to six times, Amon said, and the smartphone market is down 20% this year as a result. He said buyers are putting off upgrades rather than losing interest, saying “It’s not a demand problem” and pointing to pent-up demand. Amon put annual phone sales at about 1.2 billion units and said the market remains smaller than it was in 2019. Source: https://sources.news/p/qualcomm-ceo-ai-phone Read the article: https://llmobile.news/ticker/qualcomm-amon-100b-models-phones-2028/ChipsAgentsMemoryBusinesshttps://llmobile.news/ticker/functiongemma-mobile-actions-extended/https://llmobile.news/ticker/functiongemma-mobile-actions-extended/Sat, 26 Sep 2026 21:37:00 +0200Google’s FunctionGemma 270M-it reaches 76.5% end-to-end accuracy on mobile device commands once it is fine-tuned on a synthetic dataset, up from 29.3% for the base model on the same held-out evaluation set. The paper behind the fine-tune introduces MobileActionsExtended, a set of about 9,500 records covering 15 device-control categories. UGrowAI released the weights, the datasets, the training code and a working Android app alongside it. The 15 categories cover toggling lights, sending email and SMS, placing calls, reading battery and Bluetooth status, taking pictures and screenshots, adjusting screen brightness, and listing or opening applications. About one third of the prompts require more than one call. A second model trained jointly on the new data and Google’s own mobile actions dataset holds that extended score and reaches 82.3% on Google’s benchmark, which the authors describe as an 8 percentage point trade-off in return for doubling category coverage. The figures come from greedy decoding with no schema-constrained decoding, so the model has to emit a valid call on its own rather than being forced into the right format. FunctionGemma 270M end-to-end accuracy on mobile function calling Benchmark FunctionGemma 270M (fine-tuned) Base FunctionGemma 270M MobileActionsExtended 76.48 29.33 MobileActionsGoogle 60.15 58.79 MobileActionsExtended (combined model) 76.48 29.33 MobileActionsGoogle (combined model) 82.31 58.79 Strict exact-match end-to-end accuracy in percent on the two held-out evaluation sets, all figures according to UGrowAI and measured on unquantized checkpoints. Google's own Mobile Actions specialist scores 90.32 on MobileActionsGoogle and 17.20 on MobileActionsExtended. The demo app, Handy Chat, runs the model on a Google Pixel 8 Pro through the LiteRT-LM CPU backend rather than the GPU, with an exported bundle of 285.6 MB. The training data is synthetic rather than collected from real users, and the dataset and the demo cover only English and Android. Both fine-tuned models and both datasets are on Hugging Face under the UGrowAI account, with GitHub repositories for the fine-tuning pipeline and a downloadable APK. Source: https://arxiv.org/abs/2609.25373 Read the article: https://llmobile.news/ticker/functiongemma-mobile-actions-extended/Tool callingResearchAndroidLiteRThttps://llmobile.news/ticker/antigravity-sdk-local-models/https://llmobile.news/ticker/antigravity-sdk-local-models/Sat, 26 Sep 2026 12:58:00 +0200Google has given the Antigravity SDK the ability to drive agents with models running on the developer’s own machine, so a full agentic loop can execute without an API call. The featured configuration runs Gemma 4 26B A4B through LiteRT, and Sachin Kotwani and Taylor Mullen recommend a machine with more than 24 GB of VRAM or unified memory. Google names cost, privacy and offline resiliency as the reasons to keep the loop local. Two configuration classes carry the feature. LiteRTAgentConfig points the agent at a local LiteRT model file, while LocalOpenAIAgentConfig connects to any OpenAI-compatible server, with Ollama, LM Studio and vLLM named as examples. The SDK repository installs both with pip install google-antigravity litert-lm, states that the setup works best with gemma-4-26B-A4B-it-gpu.litertlm, and suggests a 64K context size. That checkpoint is a mixture-of-experts model, where only part of the weights is active per token, with 26 billion parameters in total and roughly 4 billion active, and it ships at about 15.8 GB as a quantisation-aware trained int4 build with float activations. A hybrid arrangement Google calls the Architect-Builder pattern splits the work between cloud and device. Gemini 3.8 Flash plans in the cloud and hands the steps to a local swarm of Gemma 4 26B instances, which Google demonstrates on a task that audits and patches three vulnerable Python modules. The run consumed 95 cloud tokens against 3,322 local ones, putting 97.2% of the tokens on the machine, with only filenames leaving it. Your browser does not support this video. This video could not be loaded. Use the link below to open it directly. Google's recording of the Architect-Builder run, with Gemini 3.8 Flash planning and local Gemma 4 instances patching the code. Open video Throughput depends heavily on the hardware. The LiteRT community model card reports 4,487 prefill and 86 decode tokens per second on an Nvidia RTX 4090 under Linux, 1,430 prefill and 77 decode on an M4 Max, and 1,676 prefill and 76 decode on an RTX 5080 under Windows, with the macOS run taking about 16.2 GB of GPU memory. The model is published under Apache 2.0, and a second example in the announcement has the local agent write a terminal dashboard with the psutil and rich libraries that tracks CPU load, memory use and the five most memory-hungry processes. Source: https://developers.googleblog.com/introducing-support-for-local-ai-models-in-the-antigravity-sdk/ Read the article: https://llmobile.news/ticker/antigravity-sdk-local-models/AgentsDeveloper toolsLiteRTGPUhttps://llmobile.news/ticker/nvidia-nemotron-3-diarization/https://llmobile.news/ticker/nvidia-nemotron-3-diarization/Sat, 26 Sep 2026 12:37:00 +0200Nvidia has released Nemotron-3-Diarization, an open-weights model with 100 million parameters that works out who speaks when in a recording, for up to eight speakers, in either streaming or offline mode. Streaming runs at an input-buffer latency of as little as 0.32 seconds, and the license is the OpenMDW License Agreement v1.1, which Nvidia states covers commercial and non-commercial use. The model takes single-channel 16 kHz audio in WAV, FLAC, OPUS or MP3, turns it into a mel-spectrogram and runs it through a 31-layer transformer encoder with rotary positional embeddings, returning per-speaker activity probabilities at 10 ms resolution in a tensor of shape [T, 8]. Streaming works through the Sortformer design, which pairs an Arrival-Order Speaker Cache holding the identities of speakers heard earlier in the conversation with a FIFO queue of recent frames. Four presets trade delay against accuracy, running from 30.4 seconds for offline processing down through 1.04 and 0.64 seconds to the 0.32-second minimum Nvidia recommends. Voice Arena Research put the model first among 12 diarization systems on its Diarization-Bench v1 leaderboard, scoring 14.7% diarization error rate at a 0 ms collar across 139 conversations and 22 hours of English audio, about 4.6 points ahead of second-placed DiariZen. Diarization error rate combines missed speech, false alarms and speaker confusion into one figure, so lower is better. Nvidia’s own measurements at the 1.04-second setting give 13.18% on DIHARD III and 7.70% on NOTSOFAR1 MHM, against 19.60% and 22.12% for the four-speaker Streaming Sortformer v2.1 it replaces. Nvidia’s own error rates at the 1.04-second latency setting, measured against its previous four-speaker model.Nvidia Local inference runs through NeMo-Speech.cpp, a native C++ runtime with CPU, Metal and Vulkan backends that loads Nvidia speech checkpoints converted to GGUF and lists this model among the ones it supports. Argmax has put the model into Pro SDK 3, its on-device speech kit for Mac and iPhone, which can be tried through the Argmax Playground app on TestFlight. The company compared six diarization systems for the release. Nemotron 3 Diarization achieved the lowest error rate (DER) across all 6 systems. Alongside the C++ runtime, the checkpoint loads in the NeMo framework and in Hugging Face Transformers through AutoModelForAudioFrameClassification, and a browser demo is on Hugging Face. The PyTorch path needs an Nvidia Ampere, Ada Lovelace, Hopper or Blackwell GPU on Linux at BF16 precision, where Nvidia reports a real-time factor of 15,113 at the offline setting and 865 at 1.04 seconds on an RTX PRO 5000 at batch size 32. Nvidia names recordings with more than eight speakers, far-field microphones, noise and reverberation as conditions that raise the error rate. Source: https://huggingface.co/nvidia/Nemotron-3-Diarization Read the article: https://llmobile.news/ticker/nvidia-nemotron-3-diarization/SpeechOpen weightsRuntimesBenchmarkshttps://llmobile.news/ticker/perplexity-portable-computer-amd-ryzen-ai-max/https://llmobile.news/ticker/perplexity-portable-computer-amd-ryzen-ai-max/Sat, 26 Sep 2026 11:58:00 +0200Perplexity has brought Portable Computer, the on-device version of its Computer agent platform, to Windows PCs running AMD Ryzen AI Max Series processors, including the Ryzen AI Halo developer platform. Systems need at least 24 GB of GPU-accessible memory to run it, and the feature is available to Pro and Max subscribers through the Perplexity Windows app. The package bundles the local model, the agent orchestrator, a scheduler, a sandbox and a sensitive-content classifier into one system instead of leaving users to assemble the parts themselves. Two local models are offered, Qwen 3.8 27B and Perplexity’s post-trained PPLX 27B, which the app downloads with one click from its Local Inference settings. AMD says its Ryzen AI Max processors combine CPU and GPU with a large pool of shared memory so local models can run alongside everyday applications. Portable Computer puts the two on-device models into the same picker the app uses for its cloud models.Perplexity Tasks run either as one-off requests or on a recurring schedule across local files and connected services such as Gmail, Outlook, Slack and GitHub, while local Model Context Protocol servers, a standard for connecting tools to models, reach other installed applications. Perplexity names financial reconciliation, daily briefings, contract review and pull request summaries as examples. Local inference does not consume Perplexity Computer credits, and escalating a task to a cloud model for current information or harder reasoning requires the user’s permission. Code runs inside an isolated sandbox whose file access is limited to folders the user approves, and an on-device classifier flags sensitive content before anything leaves the machine. Beyond the memory requirement, Portable Computer needs Windows 10 or Windows 11 and about 20 GB of free disk space. The agent first shipped on Linux and Nvidia’s DGX Spark before reaching Windows PCs with GeForce RTX and RTX PRO cards holding at least 24 GB of VRAM. Neither company published token throughput or task completion figures for Ryzen AI Max systems. Source: https://www.perplexity.ai/hub/blog/portable-computer-comes-to-amd-powered-agentic-pcs Read the article: https://llmobile.news/ticker/perplexity-portable-computer-amd-ryzen-ai-max/AgentsPrivacyGPUhttps://llmobile.news/ticker/lfm2-5-vl-dspark/https://llmobile.news/ticker/lfm2-5-vl-dspark/Fri, 25 Sep 2026 09:14:00 +0200Liquid AI has released an experimental DSpark draft model for LFM2.5-VL-3B, its 3-billion-parameter vision-language model. The drafter adds 279.5M parameters, an 8.9% increase over the deployed model. In exchange, the company reports decoding throughput gains of up to 3.13x on edge devices and up to 2.66x on GPUs, with end-to-end gains of up to 2.62x and 2.27x. On an Apple M5 Max MacBook Pro running MLX-VLM, decoding ran 2.30x to 3.13x faster across six vision tasks, with end-to-end latency improving 1.56x to 2.62x. The same tasks on an M3 Ultra with llama.cpp gave 1.57x to 2.14x faster decoding and 1.30x to 1.77x end to end, while an NVIDIA H100 80GB with SGLang reached 2.04x to 2.66x. Liquid AI measured all of it at batch size 1 and reports that the drafter’s guesses were accepted at a rate of 3.2 to 4.57 tokens per verification pass. Speculative decoding lets a small model guess the next few tokens, which the full model then checks in a single pass and accepts or rejects, so the output does not change. DSpark takes hidden states from different layers of the target model and feeds them to the drafter, a 4-layer decoder stack of 193.0M parameters combined with a 65.5M-parameter Markov head, a hidden-state projection and a confidence head. The design carries over from text to vision without modification, Liquid AI says, because image patches and text are both multi-dimensional tensors by the time they reach the hidden layers, which makes the input modality irrelevant to the drafter. The drafter receives the target model’s key-value cache and proposes a block of tokens that a hardware-aware prefix scheduler then keeps or drops.Liquid AI Gains on edge hardware stay below the raw decoding figures for a reason the company names itself. Vision encoding and prefill account for a larger share of end-to-end latency there, and speculative decoding only accelerates the decode phase. Acceleration of quantized models falls outside the scope of this release, and acceptance rates drop at higher temperatures, because probability mass spreads to lower-ranked candidate tokens where drafter and target are more likely to disagree. The weights are on Hugging Face in Safetensors and as a separate GGUF checkpoint, under Liquid AI’s LFM1.0 license, which the company describes as open-weight and free to download, fine-tune and deploy. Support ships with the release through pull requests to llama.cpp, SGLang and MLX-VLM, and the model card asks for SGLang 0.5.19 or newer on NVIDIA GPUs and MLX-VLM 0.7.2 or newer on Apple silicon. The drafter works only with the LFM2.5-VL-3B target model, at a recommended block size of 9 on NVIDIA hardware and 8 on Apple silicon. Source: https://www.liquid.ai/blog/lfm2-5-vl-dspark Read the article: https://llmobile.news/ticker/lfm2-5-vl-dspark/Open weightsBenchmarksllama.cppMLXhttps://llmobile.news/ticker/snapdragon-x2-agentic-pc/https://llmobile.news/ticker/snapdragon-x2-agentic-pc/Thu, 24 Sep 2026 08:50:48 +0200At Snapdragon Summit 2026 in Maui, Qualcomm framed the Snapdragon X2 Elite and X2 Elite Extreme, the laptop chips it launched a year earlier and has shipped since spring 2026, as the hardware for “agentic” PCs that run AI agents locally rather than answering single prompts. Qualcomm says updated configurations of the Hexagon NPU now reach up to 85 TOPS, up from the 80 TOPS specified at launch, on a platform with 228 GB/s of memory bandwidth. The company also claims up to twice the tokens per second of the first-generation Snapdragon X Elite, 450 supported on-device models, and models up to 30 billion parameters running locally with a 200,000-token context window. Much of that agent support runs through GenieX, an open-source, BSD-3-Clause inference runtime Qualcomm released in developer preview after acquiring Nexa AI earlier in 2026. GenieX accepts GGUF models from Hugging Face through llama.cpp, precompiled bundles from Qualcomm AI Hub, or a developer’s own model, and dispatches them across the Hexagon NPU, Adreno GPU or CPU. Developers reach it through a local server, a CLI, a Python SDK, Docker or Linux builds, or an Android interface. Qualcomm says AnythingLLM and LLMWare already run natively on Snapdragon X2, while OpenClaw runs directly on top of GenieX. In its compute keynote at the summit, Qualcomm said its Concierge demo uses this dispatch model for tiered routing: Clairvoyance AI sends chat and simple requests to a 3B model, light tool use to a 9B model, and high-compute agentic work to a 27B model, all served locally on the Hexagon NPU through GenieX. These figures come from the keynote presentation rather than a published spec sheet, and Qualcomm did not disclose which 27B model it used, its quantization, the RAM configuration, context length, tokens per second or sustained thermals for that demo, so the on-device feasibility of the largest tier remains unverified. One demo Qualcomm calls Creator pairs AnythingLLM with ON1 Photo RAW to search, organize and generate from a photo library using plain-language requests instead of manual tagging or folders. Qualcomm showed two further local-agent demos built with partners: a podcast assistant that uses Deepgram for real-time transcription, and a security agent built with Pokee and Halo for continuous monitoring and deepfake detection. Running large models locally on a laptop currently means choosing between different platform approaches: Apple pairs its GPU with a large pool of unified memory, AMD’s Strix Halo chips do the same, Nvidia’s laptop GPUs run faster but are limited by fixed VRAM, and Qualcomm’s Snapdragon X2 leans on NPU-driven inference backed by its platform’s 228 GB/s of memory bandwidth rather than a larger GPU. Qualcomm also published an early developer preview of Linux for Snapdragon X2 laptops, making Linux a third operating system option alongside Windows and Google’s Googlebook platform. The preview boots over UEFI, supports USB and PCIe, and drives the display and GPU through the open-source Freedreno, Turnip and Rusticl stack, while the Hexagon NPU is reachable through the FastRPC driver, running on a Debian 13 userspace with a custom-built kernel that developers have to compile themselves. Qualcomm targets foundational Debian support by the end of 2026, with Ubuntu certification and Linux laptops from HP, ASUS and HUMAIN planned for the first half of 2027. GenieX and the Linux preview are both available to developers now, though neither is consumer-ready, Microsoft’s Windows 11 26H1 update will be exclusive to Snapdragon X2 PCs, and the chips themselves have been shipping in laptops since spring 2026. Source: https://www.qualcomm.com/news/onq/2026/09/snapdragon-summit-agentic-ai-pcs-linux Read the article: https://llmobile.news/ticker/snapdragon-x2-agentic-pc/ChipsNPUDeveloper toolsOpen sourcehttps://llmobile.news/ticker/transformers-gguf-metal/https://llmobile.news/ticker/transformers-gguf-metal/Wed, 23 Sep 2026 21:56:00 +0200Hugging Face added native support in Transformers for GGUF, the quantized file format used by llama.cpp, so developers can load those checkpoints through the library’s standard from_pretrained() API instead of switching to a separate runtime. Engineers Marc Sun, Arthur Zucker and Lysandre wrote in the announcement that on Apple Silicon, weights stay packed in their GGUF blocks and Metal GPU kernels run the matmuls directly on the compressed data, skipping a full dequantization step. The feature currently covers Qwen3.5, including its dense and MoE variants, plus compatible Qwen3.8 checkpoints. The packed inference path draws its Metal kernels from a kernels library that fetches them from the Hub, including ggml-attn, the same flash attention implementation llama.cpp runs for both prefill and decode. Separate kernels handle quantization, normalization and the gated delta net used in the hybrid attention layers of both Qwen3.5 and Qwen3.8. Architectures outside this initial set, or devices other than Apple Silicon, fall back to a legacy loader that dequantizes GGUF weights into a dense model at load time, a path that already covers Llama, Mistral, Qwen2, Qwen2Moe, Phi3, Bloom, Falcon, StableLM, GPT2 and Starcoder2, among others. On a MacBook Pro M2 Max with 32 GB of memory, Hugging Face reports about 70 tok/s generating with Qwen3.5-4B at Q4_K_M quantization, close to the 72 tok/s the company measured for llama.cpp on the same hardware. The company says its new layer kernels add 51 to 109 percent more throughput over the packed-quantization-only path, depending on the model, with Qwen3.5-4B gaining 59 percent, Qwen3.8-27B 51 percent and Qwen3.5-35B-A3B 109 percent. Further gains come from generation-loop changes such as dropping unneeded attention masks early and checking stop conditions asynchronously. Quantization options for the 4B model range from a 2.74 GB Q4_K_M file up to an 8.42 GB BF16 file. The packed path requires Transformers’ main development branch together with a matching version of the kernels library, and Hugging Face says it is limited to MPS devices for now, with padding and batched requests still needing optimization. Models can also be served through an OpenAI-compatible API with transformers serve, which identifies a GGUF checkpoint as <repo>:<file>.gguf since a single Hub repository can hold several quantizations of the same model. Source: https://huggingface.co/blog/transformers-llama-cpp-quants Read the article: https://llmobile.news/ticker/transformers-gguf-metal/QuantisationGPUBenchmarkshttps://llmobile.news/ticker/apple-mac-mini-mac-studio-m5/https://llmobile.news/ticker/apple-mac-mini-mac-studio-m5/Wed, 23 Sep 2026 09:35:00 +0200Apple has started shipping an updated Mac mini and a new Mac Studio topping out at the M5 Ultra chip. Apple says the configuration lets “AI researchers run enormous LLMs entirely on device,” citing up to 512GB of unified memory and 1.2TB/s of memory bandwidth on the top Mac Studio. The Mac mini starts with a 12-core M6 chip, a 12-core GPU with Neural Accelerators in every core, and a dual 16-core Neural Engine, running 16GB of unified memory at up to 170GB/s of bandwidth. A Mac mini with the M5 Pro chip steps up to an 18-core CPU, a 20-core GPU and up to 64GB of memory at 307GB/s. Apple prices the M6 Mac mini from $899 and the M5 Pro model from $1,699. The Mac Studio comes in M5 Max and M5 Ultra configurations, with the Max offering an 18-core CPU and up to a 40-core GPU with up to 128GB of unified memory, starting at $2,499. The M5 Ultra model reaches a 36-core CPU, an 80-core GPU and up to 512GB of memory, starting at $5,499. Apple says Thunderbolt 5 with Remote Direct Memory Access lets multiple Mac Studio systems cluster together, giving up to 3x faster performance for distributed AI inference compared with a single system. The new Mac mini and Mac Studio models became available on September 22, running macOS 27 with Apple Intelligence and Siri AI features. Apple also offers leasing plans starting at $48.99 a month for the M5 Max Mac Studio and $110.10 a month for the M5 Ultra model over 36 months. Source: https://www.apple.com/newsroom/2026/09/the-new-mac-mini-and-mac-studio-are-available-today/ Read the article: https://llmobile.news/ticker/apple-mac-mini-mac-studio-m5/ChipsMemoryhttps://llmobile.news/ticker/mediatek-dimensity-cx-c10-max/https://llmobile.news/ticker/mediatek-dimensity-cx-c10-max/Wed, 23 Sep 2026 09:25:00 +0200MediaTek has launched the Dimensity CX C10 Max, the first chip in a new “Compute Experiences” line built for Googlebook, Google’s laptop category, alongside chips from Intel and Qualcomm. Built on a 3nm process, it pairs an eight-core Arm CPU and an 11-core GPU with MediaTek’s NPU 890, which the company rates at up to 55 TOPS. The CPU uses an all-big-core layout with one Cortex-X925 core at up to 3.73 GHz, three Cortex-X4 cores and four Cortex-A720 cores, backed by 12 MB of L3 cache and 10 MB of system-level cache, according to MediaTek’s product page. The 11-core Arm Immortalis-G925 GPU supports LPDDR5X memory at up to 9,600 Mbps and UFS 4.0 storage. MediaTek pitches the chip for on-device agentic AI, including multimodal, real-time workflows such as live transcription and summarization that it says run locally without sending data to the cloud. A separate Imagiq 1090 image processor supports sensors up to 32MP. Tom’s Hardware reports that the C10 Max’s CPU, GPU and NPU configuration is nearly identical to the Kompanio Ultra 910 already used in Chromebook Plus devices. The outlet notes the Cortex-X925 clock rises only from 3.62 GHz to 3.73 GHz, LPDDR5X speed from 8,533 to 9,600 Mbps, and NPU throughput from 50 to 55 TOPS, despite MediaTek’s framing of the chip as elevating its computing portfolio. Vince Hu, MediaTek’s senior vice president, said the chip is “designed for the next wave of intelligent computing, combining flagship performance and power-efficient processing.” MediaTek’s release names no pricing or launch date for Googlebook devices using the C10 Max, saying only that it enables thinner, lighter, fanless laptop designs with all-day battery life. Source: https://www.mediatek.com/press-room/mediatek-dimensity-cx-c10-max-offers-flagship-performance-to-a-new-era-of-ai-first-laptops Read the article: https://llmobile.news/ticker/mediatek-dimensity-cx-c10-max/ChipsNPUAndroidhttps://llmobile.news/ticker/snapdragon-8-elite-extreme-gen-6/https://llmobile.news/ticker/snapdragon-8-elite-extreme-gen-6/Wed, 23 Sep 2026 09:15:00 +0200Qualcomm has announced two flagship mobile chips, the Snapdragon 8 Elite Gen 6 and a pricier Snapdragon 8 Elite Extreme Gen 6, both built on a 2nm process and positioned around what the company calls the “agentic AI” era. The Extreme model’s Hexagon NPU carries 50% more shared memory than the standard chip, which lets it run mixture-of-experts models, where only part of the model activates per token, with more than 30 billion parameters directly on a phone, according to Computerbase. Qualcomm calls the chips’ Oryon prime cores, which reach 5.0 GHz, the first smartphone CPU cores to cross that clock speed, according to Tech Advisor. The fourth-generation Oryon CPU pairs two prime cores at 5.0 GHz with six performance cores at 4.0 GHz, sharing a single 16 MB cache pool that replaces the previous generation’s split 12+12 MB layout. Qualcomm claims 10% higher CPU performance on the Elite and 13% on the Extreme. The Adreno GPU clock rises to 1.45 GHz from 1.2 GHz, with reported gains of 35% and 44%, and the Extreme adds Matrix Cores for a feature called Adreno Neural Fusion, which handles AI-based upscaling and frame generation that the standard Elite lacks, according to Xenospectrum. The Hexagon NPU keeps the same 12 scalar and 8 vector units as the previous generation but adds a new tensor unit and a redesigned Element Accelerator for element-wise tensor operations. Xenospectrum reports NPU performance gains of 35% on the Extreme and 14% on the standard Elite, with efficiency improvements of 33% and 20% respectively. Qualcomm frames the extra NPU memory and efficiency around “everyday agents” that respond faster, adapt to the user and learn throughout the day, with an enhanced Sensing Hub handling always-on, on-device voice processing. Honor, Motorola, OnePlus, Oppo, Vivo and Xiaomi are confirmed adopters, while Samsung was notably absent from Qualcomm’s list. Tech Advisor names the Motorola Signature 27 as the first confirmed device. The chips also add support for LPDDR6-10600 memory alongside existing LPDDR5X-10600, and UFS 5.0 storage that doubles per-lane transfer rates to 46.4 Gbit/s, according to Computerbase. Qualcomm's own figures for the Snapdragon 8 Elite Extreme Gen 6, including the Element Accelerator's 30B+ parameter MoE support and the 5.0 GHz Oryon CPU. Credit: Qualcomm Source: https://www.qualcomm.com/news/releases/2026/09/snapdragon-leads-the-agentic-ai-age-with-two-of-the-world-s-fast Read the article: https://llmobile.news/ticker/snapdragon-8-elite-extreme-gen-6/ChipsNPUAndroidhttps://llmobile.news/ticker/googlebook-snapdragon-x-elite/https://llmobile.news/ticker/googlebook-snapdragon-x-elite/Tue, 22 Sep 2026 12:41:09 +0200Qualcomm says the Snapdragon X Elite powers the first Googlebook laptops, Google’s new laptop category built around its Gemini Intelligence features. According to Qualcomm’s release, the chip provides all-day battery life, NPU-powered tools and connectivity with Android phones. Google’s own announcement says Googlebook runs millions of Android apps, many of them optimized for large screens, mouse and keyboard input. The Snapdragon models are the Dell XPS Googlebook 13 and the HP Googlebook 14. The Snapdragon X Elite has 12 Oryon CPU cores and a Hexagon NPU rated at 45 TOPS, according to Tom’s Hardware, which Qualcomm has called the fastest NPU for laptops. Dell says the XPS Googlebook runs its local Gemini Intelligence features, including Magic Pointer and Rambler, on that NPU. Magic Pointer brings up Gemini when the user wiggles the cursor over something on screen, and features such as Cast My Apps and Continue On hand tasks and phone apps over between an Android phone and the laptop. Pre-orders are open, with devices reaching US shelves on October 4 and Canada, the UK, Ireland, France, Germany and Australia a day later, according to Google. Dell prices the XPS Googlebook 13 with 16 GB of RAM and 512 GB of storage starting at $999, while HP prices the Googlebook 14 starting at $1,199.99. Acer, ASUS and Lenovo also build Googlebooks, and Google puts the lineup’s entry price at $899. Google's own lineup image for the Googlebook launch, showing devices from several of its five hardware partners. Credit: Google Source: https://www.qualcomm.com/news/releases/2026/09/snapdragon-x-series-powers-googlebook--the-first-laptops-designe Read the article: https://llmobile.news/ticker/googlebook-snapdragon-x-elite/ChipsNPUAndroidhttps://llmobile.news/ticker/vivo-bluecode/https://llmobile.news/ticker/vivo-bluecode/Sat, 19 Sep 2026 12:19:04 +0200vivo has released a preview version of BlueCode, a native coding agent that lets users complete software development tasks on the phone itself. The company presented it at its developer conference in Shenzhen and describes the goal as turning phones from content consumption terminals into productivity tools. The reports name no supported devices, release date or technical details for the preview. vivo is also researching a 30B-parameter BlueLM MoE model for on-device use, according to Tencent News. MoE means the model is split into expert sub-networks and only part of it is active per token. Vice President Zhou Wei said the research aims to explore trillion-scale model applications on phones, and that memory and compute are the main barriers to running large models on them. If model capability improves substantially, each load would need only about 3 to 12 GB of memory, the report quotes him as saying. The 30B model sits next to the BlueLM lineup that vivo announced at the same event, which has four models for speech (BlueLM-RealTime), on-device use (BlueLM-Nano) and the cloud (BlueLM-Flash and BlueLM-Pro). According to NetEase, BlueLM-Nano handles local perception and personal memory on the device. vivo also launched OriginOS 7, and BlueOS 4 arrives on the vivo WATCH 6. Source: https://developers.vivo.com/product/ai/bluecode Read the article: https://llmobile.news/ticker/vivo-bluecode/Developer toolsBusinesshttps://llmobile.news/ticker/oppo-coloros-17-on-device-model/https://llmobile.news/ticker/oppo-coloros-17-on-device-model/Sat, 19 Sep 2026 12:19:04 +0200OPPO presented ColorOS 17 at its developer conference in Zhuhai, with an on-device large language model built on what the company calls its first Linear Attention architecture. Linear attention keeps compute cost proportional to the length of the input instead of growing quadratically. OPPO says the model supports 128K context natively, uses 48% less memory and 55% less energy than the previous generation, and keeps user data on the device while responding faster. The on-device model is part of AndesGPT — OPPO’s proprietary large model trained independently and available in sizes from around 1 billion to over 100 billion parameters, deployed in OPPO’s “1+N” end-cloud architecture (a central large model plus specialized smaller models per task). AndesGPT was the first Chinese on-device LLM to register with the Cyberspace Administration on July 15, 2026, alongside Apple Intelligence, Huawei Xiaoyi, vivo BlueLM, Xiaomi HyperAI, Samsung Galaxy AI and ZTE Nubia Doubao. The update also brings Persona X, a memory engine that builds on data, environmental and behavioral perception and records, forgets and reflects on what it learns about the user. The Xiao Bu assistant (Breeno outside China) gets longer context and better intent recognition. OPPO reports a 19% gain in internal satisfaction testing with device-cloud collaboration, and the assistant can now open 200 Alipay services, plan routes in Tencent Maps and place WeChat messages and calls by voice command. ColorOS 17 ships first on the upcoming Find X10 series and OnePlus 16, and OPPO, OnePlus and Realme are upgrading in sync. According to Gizmochina, the rollout to existing devices in China starts on October 8 with the Find X9 series and Find N6. The Find X8 series, Find N5 and tablets follow on October 16, and older flagship and mid-range devices from November 26. No global timeline has been announced. Source: https://www.jjckb.cn/20260918/447b6648587442e4b8e7b6594720fe3d/c.html Read the article: https://llmobile.news/ticker/oppo-coloros-17-on-device-model/MemoryAndroidhttps://llmobile.news/ticker/arm-mali-g2-ultra-nx/https://llmobile.news/ticker/arm-mali-g2-ultra-nx/Fri, 18 Sep 2026 20:26:52 +0200Arm has introduced the Mali G2-Ultra NX, a smartphone GPU that places dedicated neural accelerators inside each shader core. The accelerators reuse the GPU’s memory system, coherent caches and control structures, support INT8 and INT16 processing and include hardware-accelerated optical flow for motion estimation. Arm calls it the first AI-native Mali GPU and positions the accelerators for neural graphics at 1 W. The GPU is part of the Arm CSS for Mobile 2 platform. The GPU reaches up to 24% higher benchmark performance than the previous generation and 14% higher performance in non-AI gaming workloads, Arm says. Its neural graphics features are Neural Super Sampling, which reconstructs a higher-resolution image from a lower-resolution render, Neural Frame Rate Upscaling, which generates intermediate frames, and Neural Super Sampling and Denoising, which combines upscaling with denoising for ray-traced scenes. Arm says frame rate upscaling supports up to 120 FPS for longer gaming sessions. In its Neural Dawn demo, Arm reports up to 4x higher performance efficiency and up to 70% lower external memory traffic. Arm's overview slide for the Mali G2-Ultra NX Credit: Arm Arm's diagram of how Neural Frame Rate Upscaling builds an intermediate frame Credit: Arm The new execution engine is the largest update to the Mali instruction set architecture in seven generations, Arm says, with up to 2x more registers per warp. The third-generation hardware ray tracing unit adds support for Opacity Micromaps, which handle complex transparent geometry. Arm reports up to 13% lower DRAM traffic on ray tracing benchmarks, a 30% higher frame rate and up to 70% less ray tracing work in a scene from Moku’s Central Garden. For developers, Arm offers the Arm Neural Graphics Development Kit with machine learning extensions for Vulkan, plug-ins for Unreal Engine, an SDK for custom engines and tools for profiling, training and model optimization. Arm names integrations with Tencent Games Central Tech’s Magic Dawn engine, Unity China’s Tuanjie Engine and Unreal Engine MegaLights. Keli Zhou, engine lead for Where Winds Meet, says the game will be among the first to bring Arm Neural Technology to players. Arm’s post names no launch dates for devices with the GPU. Source: https://newsroom.arm.com/blog/arm-mali-g2-ultra-nx-ai-native-mobile-graphics Read the article: https://llmobile.news/ticker/arm-mali-g2-ultra-nx/ChipsAndroidhttps://llmobile.news/ticker/arm-css-for-mobile-2/https://llmobile.news/ticker/arm-css-for-mobile-2/Fri, 18 Sep 2026 20:26:52 +0200Arm has introduced Arm CSS for Mobile 2, a compute platform for smartphone chips that combines the C2 CPU cluster, the Mali G2-Ultra NX GPU and the SI L2 system interconnect. The C2 cluster pairs C2-Ultra and C2-Pro CPUs with two SME2 units, the Scalable Matrix Extension 2 that speeds up matrix math for AI on the CPU. Arm says this doubles the SME2 capability of the previous-generation configuration and reports up to 1.7x performance across the latest AI models. The cluster delivers up to 15% higher single-thread performance, 15% faster web browsing, 12% faster app launch and 12% higher multi-thread performance, Arm reports. For AI, it cites a peak uplift of up to 70% in selected tasks. Its slide compares speech, personal memory retrieval and prefill, the phase where a model reads the prompt, against the C1-Ultra with SME2. In a representative agentic workflow covering speech processing, memory retrieval, reasoning, app execution and web browsing, the C2-Ultra with two SME2 units finishes 24% faster than the previous generation, according to Arm. Arm's own comparison of the C2-Ultra with the C1-Ultra, with the AI tasks measured against the C1-Ultra with SME2 Credit: Arm The example flagship configuration in Arm’s slides has two C2-Ultra and six C2-Pro cores. The Mali G2-Ultra NX GPU integrates neural accelerators into its shader cores and adds a new execution engine and a third-generation ray tracing unit for neural graphics. Arm says the SI L2 interconnect provides lower-latency access, higher bandwidth, coherency and quality-of-service controls for CPU, GPU and other resources working at the same time. Partners can use each component on its own or combine them with custom and third-party IP. Arm's slide for the C2-Ultra CPU with an example flagship cluster layout Credit: Arm On the software side, Arm lists KleidiAI, its optimized libraries for Arm CPUs including SME2 paths, plus integrations with common AI frameworks. The Arm AI Portal offers validated models with performance and accuracy data, code examples and deployment resources, and the Arm MCP Server connects them to agentic development tools. vivo says it is bringing Arm Neural Technology to its latest flagship smartphones built on the platform, aimed at mobile gaming. Arm’s post names no launch dates for devices with CSS for Mobile 2. Source: https://newsroom.arm.com/blog/arm-css-for-mobile-2-and-c2-cpu-cluster Read the article: https://llmobile.news/ticker/arm-css-for-mobile-2/ChipsAndroidhttps://llmobile.news/ticker/arm-create-china-2026/https://llmobile.news/ticker/arm-create-china-2026/Fri, 18 Sep 2026 20:26:52 +0200Arm has published five developer takeaways from Arm Create, its developer events in Shanghai and Shenzhen. Two of them concern on-device AI. Arm says model choice starts with the workload and not with model size alone, and that developers should decide which parts of an application stay on the device, which run on nearby edge infrastructure and which need the cloud. The Shenzhen panel included Alibaba Qwen, ModelBest, Tencent Hunyuan and Ultralytics. In the Shanghai keynote, Shantu Roy, Arm’s VP of Developer Relations, discussed the Arm AI Portal. Arm says the portal lists models validated and optimized for Arm-based platforms, together with performance data for specific targets, code and deployment workflows. Coding agents can reach the same information through the Arm MCP Server. The recap shows the portal’s evaluation of Qwen3-TTS 0.6B Custom Voice, a multilingual streaming text-to-speech model from Alibaba, on a mobile CPU. The entry lists a vivo X300 with 8 CPU cores and 16 GB of memory, SME2, the XNNPACK and KleidiAI optimizations, FP16 weights and the LiteRT runtime. It reports a real-time factor of 1.2x against a baseline of 0.28x and a median end-to-end latency of 3,878 ms against 16,877 ms. Peak memory is 4,727 MB against 6,718 MB, and the evaluation uses the English subset of the MiniMaxAI TTS-Multilingual-Test-Set. Evaluation results in the Arm AI Portal, as shown in Arm's recap Credit: Arm Arm also points to Arm CSS for Mobile 2, which combines the Arm C2 CPU Cluster with SME2 and the Mali G2-Ultra NX GPU. Arm says the platform supports new on-device AI experiences on mobile. The next Arm Create event moves to the US, and Arm has not given a date. Source: https://newsroom.arm.com/blog/takeaways-from-arm-create-china-2026 Read the article: https://llmobile.news/ticker/arm-create-china-2026/ChipsAndroidTTShttps://llmobile.news/ticker/mediatek-dimensity-9600-pro/https://llmobile.news/ticker/mediatek-dimensity-9600-pro/Fri, 18 Sep 2026 17:55:18 +0200MediaTek announced the Dimensity 9600 Pro, a flagship smartphone chip built on a 2nm process. According to the company, its NPU 1090 supports on-device applications with models of up to 30B parameters. MediaTek reports 51% higher LLM prefill performance, the phase where the model reads the prompt, and 55% higher token generation per watt, measured on demo devices in its own labs. The chip pairs the NPU 1090 with a second-generation Super Efficient NPU, which MediaTek says cuts power consumption for always-on AI by 40%. The platform supports LPDDR6 memory and UFS 5.0 storage. In September 2025, MediaTek announced that it had completed the tape-out, the final design handoff to the fab, of a flagship chip on TSMC’s N2P 2nm process, with volume production expected in late 2026. The CPU uses a 2+3+3 layout of eight big cores, with two C2-Ultra cores at up to 4.55 GHz. MediaTek states up to 17% higher single-core and up to 15% higher multi-core performance over the previous generation, and up to 61% lower multi-core power consumption. The first smartphones with the Dimensity 9600 Pro and the related Dimensity 9600M are expected to launch this quarter, according to MediaTek. Source: https://www.mediatek.com/press-room/mediatek-dimensity-9600-pro-sets-new-standard-for-flagship-smartphone-chips Read the article: https://llmobile.news/ticker/mediatek-dimensity-9600-pro/ChipsAndroidhttps://llmobile.news/ticker/bonsai-2-27b/https://llmobile.news/ticker/bonsai-2-27b/Thu, 17 Sep 2026 00:00:00 +0200Prism ML has released Bonsai 2 27B, a compressed version of Qwen3.8 27B that runs on Apple devices including iPhone, iPad and Mac via MLX, using custom low-bit kernels. On NVIDIA GPUs it runs via CUDA. For the iPhone 17 Pro, Prism lists a 1-bit version at 3.9 GB with a 262K-token context window. The main Ternary Bonsai 2 27B model stores each weight as -1, 0 or +1 with FP16 scaling per group of weights. This works out to 1.76 effective bits per weight and a total footprint of 5.9 GB, which Prism says is more than 9x smaller than the full-precision model. The model accepts text and image input. Prism reports an aggregate benchmark score of 83.9 for the ternary model, compared with 85.4 for Qwen3.8 27B, which it describes as 98.2% retention. The model scores 82.66 in instruction following against 81.25 for the original, and trails it in knowledge and reasoning (83.95 against 86.66), vision (78.59 against 81.64) and agentic tool calling (77.57 against 79.74). Prism also gives 96.57 for math and 81.58 for coding, against 97.06 and 82.17 for Qwen3.8 27B. According to Prism, the ternary model generates up to 143 tok/s on an NVIDIA GeForce RTX 5090 and 46.8 tok/s on an Apple M5 Max. On an RTX 4090 it uses 0.714 mWh per token, which the company says is 40% less than a full-precision 8B model. The project repository lists peak memory for text-only use at about 7.8 GB at 4K context and 13.7 GB at 100K context, with images adding about 0.9 GB. The model is released under the Apache 2.0 license. The Hugging Face collection holds GGUF and MLX 2-bit versions, and the repository documents running it with llama.cpp on Macs with Apple silicon and on CUDA, Vulkan, ROCm and CPU. A WebGPU demo runs in the browser on Hugging Face Spaces. Source: https://prismml.com/news/bonsai-2-27b Read the article: https://llmobile.news/ticker/bonsai-2-27b/iPhonehttps://llmobile.news/ticker/cactus-needle-3/https://llmobile.news/ticker/cactus-needle-3/Thu, 17 Sep 2026 00:00:00 +0200Cactus has released Needle 3, a model with 121 million parameters for tool calling, structured extraction and text embedding. The whole model ships as a single binary of 8 to 29 MB, and Cactus says it beats models ten times its size on mobile tool calls and matches models two to three times larger on extraction. The README names mobile devices, wearables, robots, smart home, automotive and microcontrollers as targets. Needle 3 uses a Laddered Simple Attention Network, trained so that every depth from 2 to 20 layers yields a deployable model. Most of its parameters sit in an n-gram memory the README calls the engram, so the 121M model does the arithmetic of a 50M one. Weights are stored at 2.125 bits per weight in Cactus’s own quantisation format. A grammar compiled from the user’s schema constrains the output, and every response carries a calibrated confidence score next to the function calls and the model’s reasoning. The 20-layer, 121M variant scores 86.0% exact-call accuracy on Mobile Actions in the chart Cactus published with the README. That is ahead of LFM2.5 1.2B at 82.4%, Qwen3.5 0.8B at 76.0% and Needle 2 at 63.5%, while DeepSeek V4 Flash reaches 88.4%. On DroidCall the same variant scores 47.0% against 35.5% for LFM2.5 1.2B, and on BFCL v4 it scores 50.2% against 62.0% for LFM2.5 1.2B. The 8-layer, 52M variant reaches 36.8% on Mobile Actions and the 4-layer, 29M variant 11.7%. Fine-tuning on the DroidCall dataset lifts every subnetwork by 18 to 36 points, according to the README, and tuned subnetworks from 4 layers upward exceed DeepSeek V4 Flash. Fine-tuning and export run locally at 4-bit with the needle finetune and needle build commands. Cactus offers advanced 2-bit post-training through its own platform. Developers install the Python package with pip install cactus-needle and declare tools as decorated functions. Each deployment target gets a prebuilt engine under 1 MB that loads the model file at startup, with builds for macOS ARM64, Linux ARM64, the browser and WASI. The weights are on Hugging Face under Apache 2.0, according to the model card, and a browser demo is on the Cactus website. Telemetry is on by default and can be switched off with the NEEDLE_TELEMETRY=0 and DO_NOT_TRACK=1 environment variables. Source: https://github.com/cactus-compute/needle Read the article: https://llmobile.news/ticker/cactus-needle-3/Tool callingQuantisationOpen sourcehttps://llmobile.news/ticker/asus-ascent-qn10/https://llmobile.news/ticker/asus-ascent-qn10/Wed, 16 Sep 2026 00:00:00 +0200ASUS has announced the Ascent QN10, a mini PC built on the Snapdragon X2 Elite platform with an 18-core Qualcomm Oryon CPU and a Hexagon NPU rated at up to 80 TOPS. The company calls it the first AI mini PC with an 80 TOPS NPU and says it exceeds the Copilot+ PC requirements. The chassis measures under 0.7 liters, which ASUS puts at 86% smaller than a conventional 5-liter desktop, and the Adreno GPU is 2.3 times faster than in the previous generation. According to ASUS, the NPU is meant for local LLM inferencing, real-time translation, content generation, video enhancement and AI assistants. Developers can reach the Qualcomm AI Hub, which offers 175+ pre-optimized models and supports bringing their own models and data. The PC also runs agent frameworks such as OpenClaw and Hermes Agent, and ASUS describes a hybrid setup that routes workloads between cloud LLM APIs and the local NPU. Guru3d lists 16 GB or 32 GB of soldered LPDDR5X, two M.2 slots, Windows 11 on Arm and a 130 x 130 x 40 mm body, and reports that no price has been announced. The Gadgeteer names $1,349 for 16 GB with 512 GB of storage and $1,699 for 32 GB. Connectivity includes WiFi 7 and Bluetooth 6.0, and security comes from fTPM 2.0 and the Microsoft Pluton processor. Source: https://press.asus.com/news/press-releases/asus-ascent-qn10-mini-pc-snapdragon-x2-elite/ Read the article: https://llmobile.news/ticker/asus-ascent-qn10/ChipsNPUDeveloper toolshttps://llmobile.news/ticker/edge0-35b-iphone-demo/https://llmobile.news/ticker/edge0-35b-iphone-demo/Thu, 10 Sep 2026 16:08:02 +0200Samuel Zeng has released Edge0, an open-source framework for streaming mixture-of-experts inference, under Apache-2.0. In his launch post he reports a 35-billion-parameter model running on an iPhone at 1 to 2.5 GB of peak memory without a cloud connection. The repository describes three techniques. Expert weights are streamed from storage on demand, so peak memory is bounded by the active set. A trained prerouter predicts expert routing one step ahead so that loads overlap the forward pass, which the project credits with up to 59 percent higher throughput. Recover-LoRA pairs an int4-quantised frozen base with distilled LoRA adapters. Two tiers are published. edge0-35b has 40 layers and 256 experts, a checkpoint of roughly 23 GB and about 2.9 GiB of peak active memory. edge0-8b has 24 layers and 128 experts, a 4.2 GB checkpoint and about 1.0 GB peak. On a Mac mini M4 Pro with 24 GB, the project measures 14.9 to 17.7 tokens per second of decode for the 35B tier and 23.9 to 25.3 for the 8B tier, with prefill at 113 tokens per second cold and 140 warm for 35B. Quality loss against FP16 baselines is given as 3.9 points on average for the 35B tier and 2.8 points for the 8B tier. The repository states that the MLX backend runs on macOS with Apple Silicon, on M1 through M4, and requires Python 3.10 or newer. Your browser does not support this video. This video could not be loaded. Use the link below to open it directly. Demo by Samuel Zeng. Watch original on X Source: https://x.com/SamuelZengML/status/2097861839287927139 Read the article: https://llmobile.news/ticker/edge0-35b-iphone-demo/iPhoneApple SiliconOpen weightsMixture of expertsMemoryhttps://llmobile.news/ticker/llama-cpp-hexagon-npu-snapdragon/https://llmobile.news/ticker/llama-cpp-hexagon-npu-snapdragon/Thu, 10 Sep 2026 11:00:00 +0200A user has reported results from llama.cpp’s Hexagon NPU backend on a OnePlus 12 with a Snapdragon 8 Gen 3. They cross-compiled llama.cpp on Ubuntu following the project’s Snapdragon documentation and ran it from Termux on the phone. The reported figures are 8 tokens per second of prompt processing and 4.5 of generation with gemma-3-12b-it-qat at Q4_0, and 20 tokens per second of prompt processing and 12.5 of generation with gemma-3-4b-it-qat at Q4_0. According to the report the speed is about the same as running on the same chip’s CPU, but the phone does not get hot. The report lists several limitations. The backend accepts only Q4_0, IQ4_NL, MXFP4, Q8_0 and F32 files, and does not support KV cache quantisation. It runs on chips as old as the Snapdragon 888, but the poster notes that only the 8 Gen 2 and newer carry the tensor module used for language models. The Hexagon NPU can address 4 GB, so a model plus KV cache beyond that has to be split across more than one NPU device: LD_LIBRARY_PATH=./lib:/vendor/lib64 \ ADSP_LIBRARY_PATH=./lib \ GGML_HEXAGON_NDEV=2 \ ./bin/llama-completion \ -m /sdcard/gguf/gemma-3-12b-it-qat-Q4_0.gguf \ -ngl 99 --device HTP0,HTP1 \ -sys 'You are a helpful AI assistant' For context the poster cites 34 INT8 TOPS and 76.8 GB/s of memory bandwidth for the 8 Gen 3 NPU, against 80 TOPS and 228 GB/s for Qualcomm’s X2 Elite Extreme and 248 TOPS and 936 GB/s for an Nvidia RTX 3090. They ask readers with a Snapdragon 8 Elite Gen 5 phone or an X2 Elite Extreme laptop to post their own numbers, since those parts are said to address more than 4 GB and would not need multiple devices. Source: https://www.reddit.com/r/LocalLLaMA/comments/1t0k6fj/running_llamacpp_on_snapdragon_hexagon_npu_seems/ Read the article: https://llmobile.news/ticker/llama-cpp-hexagon-npu-snapdragon/AndroidNPUllama.cpphttps://llmobile.news/ticker/qualcomm-hexagon-npu-agentic-ai/https://llmobile.news/ticker/qualcomm-hexagon-npu-agentic-ai/Thu, 10 Sep 2026 00:00:00 +0200Qualcomm has teased its next-generation Hexagon NPU ahead of Snapdragon Summit 2026 (Sep 22–24), detailing an architecture redesigned around persistent on-device AI agents rather than single-pass inference. The full platform — widely expected to be a Snapdragon 8 Elite Gen 6 — will be formally announced at Summit. Qualcomm has not yet named the chip or provided absolute figures for memory capacity or NPU throughput. Two architectural changes stand out. The Element Accelerator is a new block inside the Hexagon NPU dedicated to transformer operations, working alongside the existing scalar, vector and matrix units. It targets the compute patterns that modern generative and agentic models use. A 50% larger shared memory subsystem inside the NPU keeps more model state, activations and intermediate tensors on-chip, reducing trips to external LPDDR — a move aimed at the memory wall that has long bottlenecked mobile AI, especially with long context windows. The NPU also prepares for Mixture-of-Experts (MoE) models. The company cites a 30-billion-parameter MoE that activates only about 3B parameters per token, keeping tens of billions available without the bandwidth cost of dense models. This is paired with flash-to-memory expert management and caching to load the right experts from storage. INT2, INT4, INT8, FP8 and FP16 precision are supported, and the new platform delivers up to 50% higher prefill performance for INT4 models, faster decoding throughput and enhanced speculative decoding. The design targets always-running AI with long-context reasoning and low-latency action loops, with the CPU handling routing and orchestration of multi-step workloads while Hexagon accelerates the heavy lifting. Adreno handles AI-accelerated graphics rendering and a new Neural Fusion layer extends AI acceleration into gaming. Whether this actually delivers faster, more capable local assistants depends on finished devices — several critical figures (absolute memory size, power consumption, real-world benchmarks) remain undisclosed. The full picture comes at Summit in two weeks. Source: https://www.qualcomm.com/news/onq/2026/09/hexagon-npu-agentic-ai-architecture Read the article: https://llmobile.news/ticker/qualcomm-hexagon-npu-agentic-ai/NPUAgentsMixture of expertshttps://llmobile.news/ticker/iphone-18-pro-a20-pro-neural-engine/https://llmobile.news/ticker/iphone-18-pro-a20-pro-neural-engine/Wed, 09 Sep 2026 19:00:00 +0200Apple has introduced the iPhone 18 Pro and iPhone 18 Pro Max with the A20 Pro chip. According to Apple, the chip has a new dual 16-core Neural Engine, 32 cores in total, which the company describes as double the AI processing power of A19 Pro. Apple says the Neural Engine accelerates on-device AI models and computational photography. Image: Apple. The A20 Pro also offers 50 percent more memory bandwidth than A19 Pro, and its 6-core CPU includes integrated Neural Accelerators. Apple positions the chip for “more advanced on-device AI workloads” alongside games. The phones ship with iOS 27, Apple Intelligence and the new Siri AI. Apple states that Apple Intelligence uses on-device processing together with Private Cloud Compute. Pre-orders start on 12 September, with availability from 18 September; prices start at $1,199 for the iPhone 18 Pro and $1,299 for the iPhone 18 Pro Max, both with 256 GB. Update, September 20, 2026. The retailer ZEERA reports results from a Max Tech side-by-side comparison of the iPhone 18 Pro Max and iPhone 17 Pro Max. Local LLM generation rose from 33.5 to 55.4 tok/s, a gain of 65 percent, and measured memory bandwidth from 66.5 to 111.8 GB/s, a gain of 68 percent. The comparison also lists 25 percent higher single-core and 29 percent higher multi-core CPU scores in Geekbench 7, and a 48 percent higher graphics score. Apple’s developer page for Apple Intelligence says the Foundation Models framework gives Swift apps direct access to the same on-device model that powers Apple Intelligence. New multimodal prompts pass images alongside text, and the model can call Vision framework tools such as OCR and barcode readers, all on-device according to Apple. Apps can also work with any model that conforms to the Language Model protocol, including cloud models like Claude and Gemini. Members of the App Store Small Business Program with fewer than 2 million first-time downloads get the next generation of Apple Foundation Models on Private Cloud Compute at no cloud API cost. The new Siri AI is a three-tier hybrid system, not fully on-device. A new System Orchestrator in iOS 27 routes every request to the right tier: Tier 1 — On-Device: Simple tasks (timers, app launches, local context search) run entirely on Apple’s Foundation Models (~3B parameters, 4-bit quantized) on the Neural Engine. Nothing leaves the iPhone. Tier 2 — Private Cloud Compute: Moderately complex requests route to Apple’s own servers on Apple Silicon (M2 Ultra, now M5) in hardware-isolated Secure Enclaves. Personal data is stripped and tokenized. Apple retains control; no data retention. Tier 3 — Google Gemini: The heaviest reasoning, real-time knowledge, and multimodal tasks are sent through PCC’s privacy proxy to a custom 1.2-trillion-parameter Gemini model on Nvidia Blackwell B200 GPUs in Google Cloud. Before any query reaches Google, the user’s identity and personal data are removed. Google is contractually barred from training on these queries. Apple reportedly pays Google ~$1 billion per year for this partnership. The System Orchestrator decides the routing threshold in real time based on task complexity and context size. Apple’s public materials only name On-Device and PCC as the two processing locations — Gemini is never mentioned on Apple’s consumer pages. Foundation Models can now work with any language model, including Apple’s own, Claude, and Gemini, through a single Swift interface. Apple did not publish details of the routing logic or which types of queries are sent off-device. Update, September 21, 2026. Adrien Grondin, who works on the Locally AI app, says on X that the iPhone 18 Pro runs a 27B model at double the speed of the iPhone 17 Pro. His 11-second video shows the demo, but the post names neither the model nor its quantization and gives no tok/s figures. Wccftech reports on the same demo, attributing it to Grondin and saying the model is Bonsai 2 27B from Prism ML — a ternary-weight version of Qwen3.8 27B at 1.76 bits per weight, 5.9 GB total. PrismML describes Bonsai 2 27B as released under Apache 2.0, running on Apple devices through MLX and on NVIDIA GPUs through CUDA. The iPhone 18 Pro is much better than what I expected for on-device AI It can run a 27B model at double the speed of the 17 Pro The new A20 Pro chip is a beast pic.twitter.com/gb11TvBfn0 — Adrien Grondin (@adrgrondin) September 19, 2026 Source: https://www.apple.com/newsroom/2026/09/apple-debuts-iphone-18-pro-and-iphone-18-pro-max/ Read the article: https://llmobile.news/ticker/iphone-18-pro-a20-pro-neural-engine/iPhoneChipsNPUhttps://llmobile.news/ticker/adk-for-kotlin-1-0/https://llmobile.news/ticker/adk-for-kotlin-1-0/Wed, 09 Sep 2026 00:00:00 +0200Google has released ADK for Kotlin 1.0, the general-availability version of its Agent Development Kit for Kotlin, Java and Android. The kit is built on Kotlin Multiplatform and has full feature parity with ADK 1.0 Core, according to Google. It adds a set of Android extensions for agents that run on the device. The LiteRT-LM module runs open models such as Gemma on the device with tool calling, on Android and on the JVM, and needs JDK 21 or newer according to the project README. A separate ML Kit module runs the built-in Gemini Nano model through the ML Kit GenAI Prompt API. It is Android-only, ships as a 1.0.0-beta and does not support tool calling yet. The documentation says agents can run without an API key and without a network connection at inference time, and developers can combine on-device and cloud models in one multi-agent system, with cloud Gemini models reached through Firebase AI Logic. Further Android modules store chat sessions in a local Room database, keep long-term memory in an on-device AppSearch index and save agent files in app-private storage. Skills, which are bundles of domain-specific prompts and tools, can ship inside the APK’s assets folder. Tools are declared with @Tool and @Param annotations, and a Kotlin Symbol Processing step generates their schemas at compile time with no runtime reflection. ADK for Kotlin is available under the Apache 2.0 license on GitHub, and the documentation covers setup. The repository includes runnable examples, among them a Java interoperability sample. Source: https://developers.googleblog.com/announcing-adk-for-kotlin-10-building-production-ready-ai-agents-in-kotlin-android-and-beyond/ Read the article: https://llmobile.news/ticker/adk-for-kotlin-1-0/RuntimesAndroidhttps://llmobile.news/ticker/desert-ant-labs/https://llmobile.news/ticker/desert-ant-labs/Tue, 08 Sep 2026 00:00:00 +0200Desert Ant Labs, a newly launched company, has introduced 18 on-device models as its first release, each built for a single task such as transcription, audio enhancement, personal data masking, language detection and video summarization, according to its launch post. Twelve of the models are stable and six are in beta. The first collection of Desert Ant Labs models, as shown in the launch post. Credit: Desert Ant Labs Voz, the transcription model, converts 10 minutes of audio in 2 seconds on an iPhone, according to the company. Desert Ant Labs reports 298x realtime speed on an iPhone 17 Pro and 319x on an M3 Ultra, and says Voz is 4.7x faster than Whisper and returns word-level timestamps. The audio enhancement model Clear is 9 MB and processes a 5-minute recording in 1 second, at 302x realtime on an iPhone 16 Pro and 345x on a MacBook Pro with M5. Redact masks personal data in 27 languages, takes up 12 MB and detects 88.8% of personal data, the company says. Tongue identifies 84 languages in 2 MB and reaches an accuracy of 0.933 from three words. Clips turns a 10-minute video into clips in 5 seconds with a 284 MB model, which Desert Ant Labs says is 10x faster than Claude Sonnet and uses 470x less energy at the same quality. All models are free up to 100k monthly active devices, with no tokens and no logins, according to the Desert Ant Labs website. Developers integrate them through SDKs for Swift, Kotlin and JavaScript, which are published on GitHub. The models are also on Hugging Face, where browser demos let users try them, and the company also offers a command-line tool for testing. Documentation is at desertant.com/docs. Source: https://desertant.com/blog/introducing-desert-ant-labs/ Read the article: https://llmobile.news/ticker/desert-ant-labs/Developer toolsBenchmarkshttps://llmobile.news/ticker/minicpm5-2b/https://llmobile.news/ticker/minicpm5-2b/Mon, 07 Sep 2026 10:00:00 +0200OpenBMB has released MiniCPM5-2B, published on Hugging Face and dated 7 September 2026 in the project changelog. The model card describes a dense Transformer “built for on-device, local deployment, and resource-constrained scenarios”, with 2,516,756,480 parameters of which 1,981,982,720 are non-embedding. The architecture uses 42 layers with grouped-query attention at 16 query heads and 2 key-value heads, and supports a context length of 131,072 tokens. It is the second model in the MiniCPM5 series after MiniCPM5-1B, scaling the same recipe to a larger footprint. OpenBMB reports an average score of 53.9 across its evaluation dimensions, including 69.1 on LiveCodeBench, 86.5 on AIME 2025 and 68.1 on NoLiMa. The release is licensed under Apache-2.0 and ships in BF16, GGUF, GPTQ 4-bit, MLX for Apple Silicon and LiteRT-LM for mobile, plus a DSpark draft model for speculative decoding. Source: https://huggingface.co/openbmb/MiniCPM5-2B Read the article: https://llmobile.news/ticker/minicpm5-2b/Open weightsQuantisationLiteRThttps://llmobile.news/ticker/palmclaw-android-agent/https://llmobile.news/ticker/palmclaw-android-agent/Thu, 03 Sep 2026 00:00:00 +0200Modality Dance has released PalmClaw, an open-source framework that manages sessions, memory, skills, tools and the agent loop directly on an Android phone. It exposes device capabilities as structured tools with explicit arguments and defined execution boundaries instead of driving the screen through GUI actions. The paper is listed as an EMNLP 2026 system demonstration. The language model does not run locally. In the evaluation, a Xiaomi Redmi phone with Android 13 calls DeepSeek-V4-Flash through a remote API. On 70 adapted phone tasks covering calendar, weather, contacts, audio, notes, files, Bluetooth and media, PalmClaw reaches 97.1% task success, an 11.5% relative gain over the strongest baseline. Tasks take 17.7 seconds on average, a 94.9% reduction, and need 2.8 actions each against 103.9 for the GUI-based ApkClaw. The exposed tools cover device status and permissions, media, Bluetooth, calendar, contacts, workspace files, web access, memory, communication and scheduled automation. On a 19-task AssistantBench subset for web information seeking, PalmClaw scores 36.85%. The authors name a modest task set without third-party GUI workflows, a single model backbone and one run per task as limitations. The code is on GitHub and the paper carries a CC BY-NC-SA 4.0 license. Source: https://arxiv.org/abs/2607.13027v2 Read the article: https://llmobile.news/ticker/palmclaw-android-agent/ResearchAndroidOpen sourceTool callinghttps://llmobile.news/ticker/gemini-nano-4-first-devices/https://llmobile.news/ticker/gemini-nano-4-first-devices/Thu, 27 Aug 2026 16:00:00 +0200Google’s ML Kit GenAI documentation now lists the first devices running nano-v4, 9to5Google reports. The list covers the Pixel 11, Pixel 11 Pro, Pixel 11 Pro XL and Pixel 11 Pro Fold, plus Samsung’s Galaxy Z Flip8, Galaxy Z Fold8 and Galaxy Z Fold8 Ultra. The same documentation sets Nano v3 or greater as the requirement for Gemini Intelligence, Google’s on-device feature set. According to the report, that requirement first appeared in May 2026, was removed, and has now been reinstated. Listed hardware requirements include 12 GB or more of RAM, a qualified flagship system-on-chip, five or more OS upgrades and six years of security support. Gemini Intelligence features named in the report include Rambler and Proactive Assistance on Pixel 11, and task automation across more than 40 apps on Samsung’s foldables. Source: https://9to5google.com/2026/08/27/gemini-intelligence-nano-4/ Read the article: https://llmobile.news/ticker/gemini-nano-4-first-devices/Androidhttps://llmobile.news/ticker/artificial-analysis-mobile-benchmark/https://llmobile.news/ticker/artificial-analysis-mobile-benchmark/Mon, 24 Aug 2026 15:00:00 +0200 Chart: Artificial Analysis. Artificial Analysis has published a benchmark of small language models running locally on an iPhone 17 Pro. The study covers 41 quantised builds, 33 of which ran successfully on the device. Models were executed with llama.cpp at 4-bit quantisation or smaller, and had to fit within 8 GB of memory including the KV cache at 8K context. Artificial Analysis states that the inference benchmarking is operated in partnership with Liquid AI. Nanbeige4.2-3B and LFM2.5-2.6B tied for the highest intelligence score at 63. On speed and memory the two differ sharply: LFM2.5-2.6B completed the test prompt in 8.0 seconds using 2.3 GB, against 21.4 seconds and 4.0 GB for Nanbeige4.2-3B. Ornith-1.0-9B scored 62 and Qwen3.5 9B (Reasoning) 61. End-to-end generation time for a 1,024-token prompt plus 256 output tokens ranged from 0.9 seconds for LFM2.5-230M to 26.7 seconds for Falcon-H1R-7B. Peak in-test memory at 4K context ran from 0.4 GB to 6.9 GB. The intelligence index combines tool calling, instruction following, knowledge, scientific reasoning and mathematics. Artificial Analysis notes that context limits, not parameter count alone, held back several 9B models, which frequently hit a 16K ceiling. Chart: Artificial Analysis. Source: https://artificialanalysis.ai/articles/mobile-phone-intelligence-inference Read the article: https://llmobile.news/ticker/artificial-analysis-mobile-benchmark/BenchmarksiPhoneQuantisationllama.cpphttps://llmobile.news/ticker/graphcore-llama-mobile/https://llmobile.news/ticker/graphcore-llama-mobile/Mon, 24 Aug 2026 00:00:00 +0200Graphcore Research has fitted the 11-billion-parameter vision-language model Llama-3.2-11B-Vision-Instruct into 3.6 GB, down from 21 GB, using a new 2.7-bit weight format called S3D8. In the team’s blog post, the model runs on a Pixel 8a with a custom C++ inference implementation and generates 3.8 tok/s. Your browser does not support this video. This video could not be loaded. Use the link below to open it directly. Open video S3D8 stores three weights in 8 bits, which comes to 2.68 bits per parameter. Each group of three weights holds a 5-bit index into a shared table of centroids plus 3 bits of sign information, and the values are decoded to channel-scaled INT8 for compute. On the Pixel 8a using 5 cores, the fused dequantize-and-multiply kernel reaches 33.8 GMAC/s for single-token generation (GMAC/s means giga multiply-accumulate operations per second, a standard measure of AI compute throughput on devices), compared with 26.5 GMAC/s for INT8 and 13.6 GMAC/s for bfloat16, according to Graphcore. Graphcore measured the 3.8 tok/s while generating 100 tokens from a single image tile. Graphcore measured accuracy on 1,024-example subsets of VQAv2, ChartQA, DocVQA and AI2D. S3D8 averages 0.661 against 0.744 for the bfloat16 original, while a plain integer format of similar size averages 0.347 and a student-t format 0.565. The team recovered quality with quantization-aware training by distillation, where a frozen bfloat16 teacher guides the quantized student over 2,048 steps. Training data consists of ImageNet images paired with text the teacher generated from a pool of 495 prompt seeds, and the team says the wrong data lowers the training loss while the model gets worse on downstream tasks. The demo needs a Pixel 8a or newer Android phone with 8 GB of RAM and 4 to 5.5 GB of free storage, and a CPU that supports the Arm i8mm and bf16 extensions. Collaborators from Arm contributed to the work. The demo APK can be downloaded from Graphcore, the Android code is on GitHub and the method is described in a paper on arXiv. Graphcore says the model is too large and compute-heavy for the phone, so although it runs, it is not practical. Update, September 19, 2026. The paper puts the compressed model at 3.73 GB including vocabulary and metadata, of which 3,569 MB is weight storage at 2.68 bits per parameter. The authors also ran S3D8 on a Graviton4 server chip with 96 Neoverse V2 cores, where end-to-end decoding reaches 36.8 tok/s against 26.4 tok/s for INT8. On the Pixel 8a, decoding reads weights at 12.5 GB/s. Per task, S3D8 scores 0.702 on VQAv2 (bfloat16 0.754), 0.648 on ChartQA (0.747), 0.740 on DocVQA (0.844) and 0.554 on AI2D (0.631). The paper lists its limits as a single evaluated model, CPU-only execution and one fixed format for all weights, since mixed-precision experiments did not work. Training used 2,048 steps at batch size 128 on 1.28 million ImageNet images with teacher-generated responses. Source: https://graphcore-research.github.io/2026-08-24-llama-mobile/ Read the article: https://llmobile.news/ticker/graphcore-llama-mobile/QuantisationResearchAndroidOpen sourcehttps://llmobile.news/ticker/s2-moe-self-speculative-decoding/https://llmobile.news/ticker/s2-moe-self-speculative-decoding/Wed, 19 Aug 2026 00:00:00 +0200Researchers at Peking University have published S2-MoE, a self-speculative decoding framework for mixture-of-experts (MoE) models on edge devices. MoE models activate only a few experts per token, which makes verifying several drafted tokens at once expensive. The method reports a speedup of up to 5.3x, about 2.0x on average, over standard autoregressive decoding and is implemented in llama.cpp. The paper tests DeepSeek-V2-Lite-Chat (15.3B parameters, 1.35B active), OLMoE-1B-7B (6.7B, 1.1B active), Qwen3-30B-A3B (30.5B, 3.3B active) and GPT-OSS-120B (116.8B, 5.7B active). Hardware is a Jetson Orin NX 16GB, Jetson AGX Orin with 32 GB or 64 GB, and an RTX 4090 with constrained GPU memory. On the Orin NX, OLMoE rises from 2.30 to 12.08 tok/s and DeepSeek from 0.55 to 2.07 tok/s. Speedups on that board range from 2.10x to 3.17x for Qwen3 and from 1.57x to 2.53x for GPT-OSS. The draft pass uses fewer experts per token, and the draft and target share one KV cache. Routing-aware expansion uses the draft’s expert routing to estimate verification cost and adds candidates only where the expected gain is higher. Reuse-aware gating nudges the router toward frequently activated experts, and expert-level offloading handles models that do not fit in memory. The authors measure a perplexity ratio of 1.012 to 1.013 and top-1 agreement above 89%, with comparable results on LongBench. They compare against standard autoregressive decoding, layer-sparsity and expert-sparsity self-speculation, EAGLE-3 and Cascade, and say other MoE offloading work lacks public llama.cpp implementations. The code is on GitHub. Source: https://arxiv.org/abs/2608.15018v2 Read the article: https://llmobile.news/ticker/s2-moe-self-speculative-decoding/ResearchOpen sourceGPURuntimeshttps://llmobile.news/ticker/ornith-1-5/https://llmobile.news/ticker/ornith-1-5/Wed, 19 Aug 2026 00:00:00 +0200Ornith has released Ornith-1.5, a model family with a 9B dense model, a 35B mixture-of-experts model that activates about 3B parameters per token and a 397B mixture-of-experts model. The 9B model also comes as Ornith-1.5-9B-Mobile, a quantized variant (about 1,5 GB on disk) designed for iPhone and Android. The 9B model scores 47.0 on Terminal-Bench 2.1 with the Claude Code harness and 70.6 on SWE-bench Verified in Ornith’s tests. The company says the model matches or exceeds much larger models such as Gemma 4-31B and Qwen 3.6-35B. In the company’s chart, Qwen3.6-35B-A3B leads on SWE-bench Verified with 73.4 and on Terminal-Bench 2.1 with 52.5, while the 9B model scores 86.4 on GPQA Diamond and 54.2 on MCP-Atlas. The previous Ornith-1.0-9B reaches 43.1 on Terminal-Bench 2.1 in the same chart. Ornith's own benchmark figures for the 9B model. According to Ornith, its training loop lets the model propose new tasks, generate task-specific scaffolds and produce solution rollouts, with the reward from the rollouts propagated across all three stages. The company reports that the 35B model scores 67.8 on Terminal-Bench 2.1 with the Terminus-2 harness, against 52.5 for Qwen 3.6-35B. The models are on Hugging Face, with GGUF builds of all three sizes and MLX builds of the 9B and 35B models. Source: https://ornith.ai/ornith_1_5.html Read the article: https://llmobile.news/ticker/ornith-1-5/Androidhttps://llmobile.news/ticker/online-sdft/https://llmobile.news/ticker/online-sdft/Tue, 18 Aug 2026 16:57:46 +0200I-Ju Lin and Zhang-Wei Hong published Online-SDFT, a method that keeps fine-tuning a small language model on a phone from the user’s own stream of decisions. On a synthetic notification-routing benchmark with Liquid AI’s LFM2.5-230M, the authors report 70.28% of decisions matching a hidden user preference against 52.78% for the strongest baseline they tested, and cumulative regret of 44.76 against 105.26, over 720 decisions in three paired streams. They state that this comparison ran on Apple MPS rather than on a phone, and that the confidence intervals are nominal across only three streams. Self-distillation here means the model teaches itself. The model picks an action first, the phone later observes what happened, and the same model is then run again with that outcome added to its input. Its better-informed answer becomes the training target for the version that saw only the original context, so no second teacher model, no human label and no reward score is needed. The authors keep the 230M base frozen and train a LoRA adapter of 172,032 parameters across 48 tensors in six attention layers, with the teacher pass running with the adapter switched off so student updates cannot move the teacher. Feeding one interaction at a time into so small an adapter makes it noisy and prone to forgetting, so the authors keep a 64-row buffer of recent lessons and sample each batch from it. Their ablation puts that buffer ahead of the exploration schedule as the load-bearing part, with accuracy dropping from 70.28% to 39.58% and regret climbing from 44.76 to 134.90 once replay is removed, while dropping exploration instead leaves accuracy at 69.44% and regret at 62.00. Diagram: I-Ju Lin and Zhang-Wei Hong. The repository carries an Android proof of concept in which LFM2.5-230M decides and ONNX Runtime Training performs the adapter update on the device, with replay and checkpoints in app-private storage that survive a restart. Exporting the model still happens on a Linux host, and the deployment guide asks for Android 12 or newer on ARM64 with 12 GB of RAM at minimum, 16 GB preferred, and 8 GB of free storage, because the training graph is FP32. Your browser does not support this video. This video could not be loaded. Use the link below to open it directly. Video: I-Ju Lin and Zhang-Wei Hong. Watch on the project page The authors call that graph a correctness implementation rather than a claim of practical background training on low-memory phones, and state that they did not measure Android latency, peak memory, battery use or thermal behaviour, and that the benchmark uses synthetic streams rather than real users. The code, the Android app and a Colab notebook are on GitHub under the MIT license. No paper or preprint accompanies the release. Source: https://lin826.github.io/SLM-Online-SDFT/ Read the article: https://llmobile.news/ticker/online-sdft/ResearchDistillationAndroidOpen sourcehttps://llmobile.news/ticker/pixel-watch-5-offline-gemini/https://llmobile.news/ticker/pixel-watch-5-offline-gemini/Wed, 12 Aug 2026 19:30:00 +0200Gemini Intelligence is coming to the Pixel Watch 5, 9to5Google reports. According to the report, offline Gemini commands on the watch use a separate on-device model when the phone or an internet connection is unavailable. Those commands cover timers and alarms, brightness and modes, music control, opening apps and starting workouts. On-device smart replies now offer three responses instead of one, which the report attributes to a Gemini Nano upgrade that makes them 50 percent faster. Other parts of the feature set depend on a connection. Proactive Suggestions, formerly Magic Cue, are generated on a paired Pixel 11 and bridged to the watch, and Personal Intelligence draws on Gmail, Calendar and Keep. The update also brings a new At a Glance space on the watch face for timers, workouts, music, flight details and navigation. Source: https://9to5google.com/2026/08/12/pixel-watch-5-gemini-intelligence/ Read the article: https://llmobile.news/ticker/pixel-watch-5-offline-gemini/Wearableshttps://llmobile.news/ticker/pixel-11-tensor-g6/https://llmobile.news/ticker/pixel-11-tensor-g6/Wed, 12 Aug 2026 19:00:00 +0200Google has announced the Pixel 11, Pixel 11 Pro and Pixel 11 Pro XL, built around the Tensor G6 chip. Google states that Tensor G6 packs 50 percent more TPU compute and, paired with the latest Gemini Nano model, processes on-device AI tasks up to 3.5 times faster while using up to 3.5 times less energy. The company also cites an upgraded CPU with 25 percent faster web browsing and 15 percent quicker app launches, and says the chip powers the 30x Super Zoom on the 5x telephoto lens. Google does not publish RAM figures, model sizes or per-task latency in the announcement. Pre-orders opened on 12 August, with retail availability from 20 August. ▶Meet Google Pixel 11 ProLoading connects your browser to www.youtube-nocookie.com, which may process your IP address and use cookies. Privacy details Load contentOpen externallyVideo: Made by Google. Source: https://blog.google/products-and-platforms/devices/pixel/google-pixel-11-pro-xl/ Read the article: https://llmobile.news/ticker/pixel-11-tensor-g6/ChipsNPUhttps://llmobile.news/ticker/rikkahub-agent-local-llm/https://llmobile.news/ticker/rikkahub-agent-local-llm/Sat, 08 Aug 2026 00:00:00 +0200Adam Conway at XDA has tested RikkaHub Agent, a fork by ExTV of the open-source Android LLM client RikkaHub, which adds an agent layer with more than 80 device tools. The phone runs the agent loop and executes the tools; the model itself runs elsewhere on the local network over an OpenAI-compatible endpoint. The test device was an Oppo Find N5 with a Snapdragon 8 Elite and 16 GB of RAM. Inference ran on a Radeon RX 7900 XTX with Qwen 3.6 27B at Q4_K_M, roughly 16 GB, at 40 to 50 tokens per second. A second configuration used Qwen 3.6 35B on a Lenovo ThinkStation PGX with Nvidia’s GB10 Grace Blackwell. Screenshot: XDA. The tool set covers tapping, swiping, scrolling and typing on screen, reading notifications, sending SMS, file management, browser control, SSH, scheduled jobs, a JavaScript runtime and Telegram. All tools are off by default and enabled per assistant, actions with side effects require approval, and a deterministic blocklist rejects destructive commands. The default context is 7,000 input tokens, rising to 32,000 with every tool enabled. Asked whether whisper.cpp was installed, the agent determined that neither the command-line tool nor a model was present, requested permission, then installed the dependencies, cloned the repository and compiled it inside Termux. The build took seven minutes, and the agent chose the 75 MB small model on speed and accuracy grounds. It then transcribed a Telegram voice message, installing ffmpeg, converting the OGG/Opus file to 16 kHz mono WAV and returning a transcript in about two seconds. Running the model on the phone itself was slower. With Gemma 4 E2B, a 2.6 GB build accelerated by LiteRT-LM, Conway reports chat worked but agentic tasks were painful, and tool calls were emitted as plain text until an update fixed the behaviour. Frontier intelligence is never what I’d want from a phone agent, which is why a local LLM works perfectly. Source: https://www.xda-developers.com/turned-phone-ai-agent-powered-local-llm-installed-software/ Read the article: https://llmobile.news/ticker/rikkahub-agent-local-llm/AndroidAgentsOpen sourceSpeechhttps://llmobile.news/ticker/lfm2-5-2-6b/https://llmobile.news/ticker/lfm2-5-2-6b/Tue, 04 Aug 2026 17:00:00 +0200Liquid AI has released LFM2.5-2.6B, a 2.6-billion-parameter model built on its LFM2 architecture for edge deployment. The company reports 30 tokens per second on a phone, with CPU inference staying under 2.5 GB of memory. Its published chart identifies the device as a Qualcomm Snapdragon Galaxy (SM-S948U1) and puts memory use at 2,448 MB at Q4_K_M quantisation with a 4K-token input context. Chart: Liquid AI. On other CPUs Liquid AI cites 220 tokens per second on an M5 Max and 113 tokens per second on a Ryzen AI Max+ 395. The model was pre-trained on roughly 34 trillion tokens, uses a 128K vocabulary, and its mid-training included a dedicated 128K context-extension phase. Liquid AI positions the model for agentic use, describing planning, tool calling and multi-step tasks, and states that it is competitive with, and often outperforms, models nearly four times its size on instruction-following and tool use, while larger models keep an advantage in coding. Weights are on Hugging Face in BF16, GGUF for llama.cpp, and ONNX. Source: https://www.liquid.ai/blog/lfm2-5-2-6b Read the article: https://llmobile.news/ticker/lfm2-5-2-6b/Open weightsBenchmarksAgentshttps://llmobile.news/ticker/gemini-nano-4-mlkit-prompt-api/https://llmobile.news/ticker/gemini-nano-4-mlkit-prompt-api/Wed, 22 Jul 2026 18:00:00 +0200Google’s Android developer blog states that Samsung’s new foldable devices come with Gemini Nano 4, which it calls its latest on-device model. The post credits Nano 4 with support for more than 140 languages and better multimodal understanding. Apps reach the model through ML Kit’s Prompt API, which sends natural language requests on-device to Gemini Nano. It takes text, or a combination of image and text, and returns text or structured output. Google names structured output and thinking mode as the features to use for on-device intelligence. The ML Kit release notes dated 14 July 2026 record the structured output API, system instructions and thinking mode arriving in the Prompt API, along with multi-image support and an output token limit raised to 4,096 tokens. A note dated 21 July records a fix for Gemini Nano v4 compatibility in the Prompt API on non-Pixel devices. The Prompt API moved from alpha to beta in January 2026 and carries no service level agreement or deprecation policy. The post also points developers to app functions, which share an app’s capabilities with the Gemini Intelligence system. Its remaining sections cover adaptive layouts, fold-aware design, CameraX and Wear OS widgets. Source: https://android-developers.googleblog.com/2026/07/optimize-galaxy-screen-sizes.html Read the article: https://llmobile.news/ticker/gemini-nano-4-mlkit-prompt-api/AndroidDeveloper toolshttps://llmobile.news/ticker/mobibench-llamacpp/https://llmobile.news/ticker/mobibench-llamacpp/Wed, 15 Jul 2026 00:00:00 +0200Researchers have published MobiBench, a benchmarking suite built on llama.cpp that measures how small language models behave under limited resources. It covers prefill speed, decode speed, time-to-first-token, memory use and task accuracy. The paper tests Gemma 1B, TinyLlama, Llama 3B and Phi-2, most of them released in 2023 and 2024, plus Q2_K and Q3_K_M quantized variants of Phi-2. A diagram in the paper shows the llama.cpp execution pipeline from runtime initialization and model loading to compute graph construction and iterative decoding. Figure 1 of the paper, titled high-level execution pipeline of llama.cpp. Credit: MobiBench authors, arXiv Despite the mobile framing, the authors ran the tests on a gaming laptop with an Intel Core i7 and an RTX 3060 GPU and on a standard laptop with an Intel Core i5 and no dedicated graphics. The paper reports no phone, Android or ARM measurements and no battery or energy data. On the GPU, Llama 3B reaches 225 to 243 tok/s in decode, Phi-2 reaches 102 to 122 tok/s and Gemma 1B reaches 150 to 167 tok/s. The Gemma 1B figure is about 7 to 8 times the 20 to 23 tok/s the authors measured on the CPU. For TinyLlama, GPU inference cut average wall-clock time per sample from 65.81 s to 1.39 s on MMLU, a 12 to 47 times speedup across the four tasks. On the GPU, Phi-2 used about 2,070 to 2,091 MB of memory, the Q3_K_M variant about 1,982 to 2,000 MB and the Q2_K variant about 1,694 to 1,710 MB. The Q2_K variant decoded at 46 to 56 tok/s and the Q3_K_M variant at 58 to 88 tok/s. On MMLU, Phi-2 scored 0.243, the Q3_K_M variant 0.263 and the Q2_K variant 0.233. The accuracy tests used 1,000 samples each from RepLiQA, CNN/DailyMail summarization, SciQ and MMLU. Summarization BERTScore F1 stayed between 0.8388 and 0.8458 across all models, and an LLM-based semantic evaluation gave mean scores from 0.4554 for TinyLlama on the GPU to 0.6961 for Gemma 1B on the CPU. The authors list several limitations, namely the restriction to llama.cpp and consumer-grade Intel and RTX 3060 hardware, modest scores on domain-specific reasoning tasks and varying output formats that affect multiple-choice scoring. Source: https://arxiv.org/html/2609.13159v1 Read the article: https://llmobile.news/ticker/mobibench-llamacpp/BenchmarksResearchQuantisationGPURuntimeshttps://llmobile.news/ticker/bonsai-27b/https://llmobile.news/ticker/bonsai-27b/Tue, 14 Jul 2026 00:00:00 +0200Prism ML has released Bonsai 27B, a compressed version of Qwen3.6 27B for Mac, iPhone, iPad and NVIDIA GPUs. The 1-bit variant needs 3.9 GB, which Prism says fits within the memory budget of an iPhone 17 Pro. The company notes that a phone never exposes its full memory to an app and that a 12 GB iPhone offers about 6 GB to a model. The ternary variant stores each weight as -1, 0 or +1 and takes up 5.9 GB at 1.71 effective bits per weight. The 1-bit variant stores weights as -1 or +1 at 1.125 effective bits. Prism applies the low-bit format to embeddings, attention layers, MLPs and the output head. Both models accept images through a 4-bit vision tower and offer a 262K-token context window. Across a 15-benchmark suite, Prism reports an overall score of 85.0 for Qwen3.6 27B, 80.5 for the ternary model and 76.1 for the 1-bit model, which it describes as retaining 95% and 90% of the original. The largest gaps show up in agentic tool calling, at 80.0 for the original, 74.0 for ternary and 66.0 for 1-bit, and in vision, at 72.6, 65.2 and 59.6. Prism also publishes an “intelligence density” figure, defined as the negative log of the error rate divided by model size in GB, of 0.530 for the 1-bit model and 0.400 for the ternary one. On an NVIDIA GeForce RTX 5090, Prism measures 163 tok/s for the 1-bit model and 134 tok/s for the ternary model. On an Apple M5 Max the figures are 87 and 58 tok/s. The models also support speculative decoding, where a small draft model proposes tokens that the main model then verifies. Bonsai 27B is released under the Apache 2.0 license, with builds for MLX on Apple devices and for CUDA on NVIDIA GPUs. Weights are in the Hugging Face collection, and the Bonsai-demo repository covers running them. Prism also lists the Locally AI iOS app and a free, limited-time developer API on Together.ai as ways to try the model. Source: https://prismml.com/news/bonsai-27b Read the article: https://llmobile.news/ticker/bonsai-27b/QuantisationBenchmarksOpen sourcehttps://llmobile.news/ticker/fblayout/https://llmobile.news/ticker/fblayout/Tue, 07 Jul 2026 12:18:25 +0200Researchers at the University of Macau, the University of Georgia and the Hong Kong University of Science and Technology published FBLayout on July 7, 2026, a system that fine-tunes transformer models on a phone’s own GPU, accepted at ACM MobiSys 2026. On a OnePlus Ace 5 Pro with a Snapdragon 8 Elite and its Adreno 830 GPU, the authors measure training on Llama 3.2 1B, Qwen2.5 1.5B and Gemma 2 2B at 3.9 to 4.1 times the speed of MNN, 4.3 to 4.9 times TFLite and 5.4 to 5.7 times TVM. Over the same three baselines they count 3.5 times fewer global memory reads and 4.2 times fewer cache misses. The authors time inference on the Snapdragon 8 Elite at roughly 6.5 times faster on the GPU than on the CPU, while fine-tuning models of 0.5B to 2B parameters on the GPU comes out level with the CPU or up to 43 percent worse. Their explanation is memory layout, meaning the order in which a tensor’s numbers physically sit in memory. Mobile GPUs hold tensors in a 2D texture grid whose cache rewards reading neighbouring cells, and training reuses forward-pass activations and weights in the backward pass, which sums along a different dimension and so walks the same tensor in strides instead of in rows. Each stride that misses the cache becomes a fetch from DRAM at under 100 GB/s of mobile memory bandwidth, and the GPU stalls waiting for it. Existing frameworks take one of two ways out, and the paper prices both. MNN reuses the forward layout in the backward pass, which the authors put at up to a 5.3 times slowdown on key operations and at under 21 percent arithmetic-unit utilisation in a matrix-multiply benchmark. TFLite instead inserts explicit transpose and reshape operators, and the paper puts that route at 40.8 to 50.0 percent of total runtime across the three LLMs and at roughly a third of all memory allocations. FBLayout defines one tile-based layout, called R-Tile, that keeps both directions contiguous, then rewrites the remaining reshapes as coordinate arithmetic so no data moves, which they report covers over 86 percent of all layout transformations and over 95 percent of those in the LLMs. The evaluation covers seven models, adding BERT-Large, ViT-Large, Whisper-Large and Stable Diffusion 1.5 to the three LLMs, with LoRA adapters in the attention and MLP blocks for everything except ViT. Beyond the Snapdragon 8 Elite phone the authors test a OnePlus Ace 10 Pro with a Snapdragon 8 Gen 1 and an Adreno 730 and a OnePlus Ace 5 Ultra with a Dimensity 9400+ and an Arm Mali Immortalis-G925, and report gains on both. Preparing the Llama 3.2 1B training graph takes 6.1 seconds in their measurement, against 723 seconds in MNN, 862 in TFLite and over 4,800 in TVM, and a Monsoon power monitor puts energy use over five training rounds at 3.5 to 6.3 times below the baselines. FBLayout is built on top of Alibaba’s MNN as about 9.2K lines of C++ and OpenCL, extending it with automatic differentiation so the training graphs exist in the first place. The paper names no code release and points to no repository. It appeared at MobiSys 2026 in Cambridge in June 2026 and is on arXiv under a CC BY-NC-ND 4.0 license. Source: https://arxiv.org/abs/2607.21624 Read the article: https://llmobile.news/ticker/fblayout/ResearchMemoryAndroidhttps://llmobile.news/ticker/mobile-npu-llm-inference-study/https://llmobile.news/ticker/mobile-npu-llm-inference-study/Mon, 06 Jul 2026 00:00:00 +0200A team of researchers has published a cross-layer measurement study of mobile LLM inference titled “Is Your NPU Ready for LLMs?”. It covers five frameworks, llama.cpp, MNN, MLC-LLM, MLLM and Qualcomm’s GENIE, on CPU, GPU and NPU backends. The study finds that framework-induced performance gaps grow on NPUs and reach up to 10x. The tests ran on four Snapdragon phones, the Xiaomi 17, OnePlus 15, Xiaomi 15 and Xiaomi 14, with Llama 3.2 1B and 3B, Qwen 2.5 1.5B and 7B and Phi 3.5 3.8B, mostly with 4-bit weights. At a 256-token context, GENIE on the NPU reaches 1,463.7 tok/s in prefill against 115.1 tok/s for llama.cpp. In decoding the order flips, with the CPU at about 50 to 76 tok/s and the NPU at about 10 to 34 tok/s. The authors attribute this to NPUs preferring large, fixed-shape workloads, which conflicts with the small, dynamic kernels of decoding. For energy, the team built PowerBench, a lightweight C++ library that maps PMIC power zones to CPU, GPU and NPU for backend-specific attribution. They report that suboptimal configurations waste up to 40% of energy and that a tuned setup cuts NPU energy by up to 54.8%. The recommended settings are a 20 μs RPC polling interval, an NPU sleep latency of 65535 μs, the CPU at its lowest frequency and full-graph offloading with QNN. Source: https://arxiv.org/abs/2607.05475 Read the article: https://llmobile.news/ticker/mobile-npu-llm-inference-study/ResearchBenchmarksNPUAndroidRuntimeshttps://llmobile.news/ticker/gemini-nano-frozen-mtp/https://llmobile.news/ticker/gemini-nano-frozen-mtp/Fri, 26 Jun 2026 00:00:00 +0200Google Research has retrofitted multi-token prediction (MTP) onto the production Gemini Nano v3 models, which Google says recently rolled out to the Pixel 9 and Pixel 10 series. A small Transformer head attached to the last layers of the model proposes several future tokens per pass, and the unchanged main model verifies them in parallel. According to Google, MTP correctly predicts an average of nearly two additional tokens per inference pass in AI Notification Summaries and Proofread. Gemini Nano v3 stays frozen and only the MTP head is trained, so Google says the output is bit-for-bit identical to the original model and its capabilities and safety alignment do not change. The head cross-attends to the main model’s existing key-value cache instead of keeping its own, which avoids a second prefill pass. Google reports savings of 130 MB per instance compared with a standalone drafter, a separate small model that proposes tokens in classic speculative decoding, because the head needs no embedding lookup tables, prefill attention variants or application-specific tuning parameters. Google measured speedups of 50% or more on Pixel 9 depending on the task, compared with standalone drafters of comparable parameter count. For highly predictable outputs such as smart replies, token acceptance improved by up to 55%. The company says features like notification summaries, Proofread and smart replies now generate text faster and with less energy, and that developers no longer need to fine-tune a separate drafting model for each new task. Google's own figures show end-to-end latency gains of 1.14x to 1.48x across five Pixel 9 features. Credit: Google Research Google plans to bring MTP to future Pixel devices. The research team also wants to explore parallel decoding without auxiliary heads to cut draft latency, models that explore several branches in parallel, and a relaxed verification step that no longer requires an exact token match for some use cases. Source: https://research.google/blog/accelerating-gemini-nano-models-on-pixel-with-frozen-multi-token-prediction/ Read the article: https://llmobile.news/ticker/gemini-nano-frozen-mtp/ResearchAndroidhttps://llmobile.news/ticker/qualcomm-amon-agents-new-app/https://llmobile.news/ticker/qualcomm-amon-agents-new-app/Tue, 16 Jun 2026 07:08:00 +0200Qualcomm chief executive Cristiano Amon says the company is working on more than 40 designs of new AI devices, CNBC reports from an interview on its “The Tech Download” podcast. He listed jewellery, earbuds with cameras, pins and watches among the form factors. Amon described a shift in what devices are organised around: The phone is around the agent. The new classes of devices … are going to be around the agent as well. And the agent will be the one that will understand human intentions and will do things for you, so there is a shift in what the center of gravity is. He said phones will not disappear. On software, he told CNBC that apps are “not dead” but “are going to change”, adding: “Those agents are going to be the new app.” His example was an agent that retrieves banking transaction details without the user navigating the app. On smart glasses, Amon put current shipments in the “order of multiple tens of millions” a year and said that in “a couple of years” the category could reach the “order of hundreds of millions of glasses and could become as big as smartphones”. CNBC cites Counterpoint Research figures of 1.26 billion smartphones shipped in 2025, about 3 percent above the prior year. Amon linked the entry of AI companies into hardware, such as OpenAI’s purchase of Jony Ive’s startup io, to both agent endpoints and training data, saying the devices will gather data on a scale that is “exponentially larger” than the data used to train current models. He said Qualcomm’s chip roadmap is being reworked in response: “An entire roadmap, because I believe none of the devices we have today are prepared for the future.” The full interview is available as a video on CNBC. Source: https://www.cnbc.com/2026/06/16/qualcomm-ceo-ai-devices-agents.html Read the article: https://llmobile.news/ticker/qualcomm-amon-agents-new-app/AgentsWearablesGlasseshttps://llmobile.news/ticker/mlperf-inference-mobile/https://llmobile.news/ticker/mlperf-inference-mobile/Mon, 15 Jun 2026 17:00:00 +0200MLCommons announced MLPerf Mobile v6.0 on June 15, 2026, adding generative AI tests that run Llama on an Android device and report token throughput. The new workload comes in 1B, 3B and 8B parameter sizes, using Llama 3.2 1B Instruct, Llama 3.2 3B Instruct and Llama 3.1 8B Instruct. Each size runs twice, once against TinyMMLU, a small multiple-choice knowledge set, and once against IFEval, which checks whether a model obeys the instructions in a prompt, according to the task configuration in the app repository. MLCommons says the tests run on devices with enough memory through the CPU without tailored acceleration, and that v6.0 adds NPU-accelerated execution of Llama 3.1 8B Instruct on Qualcomm Snapdragon 8 Elite Gen 5. The v6.0 app release of June 9, 2026 also brings support for MediaTek Dimensity 9500 chips and updates support for the Samsung Exynos 2600. The rest of the suite covers image classification with MobileNetV4 on ImageNet, object detection on COCO 2017, segmentation with MOSAIC on ADE20K, language understanding with MobileBERT on SQuAD 1.1, super resolution with EDSR, and Stable Diffusion 1.5 on COCO captions, as the benchmark page lists them. Almost every test runs in the single stream scenario, where the score is the 90th percentile latency for one query at a time, and image classification also runs an offline pass scored as throughput. Stable Diffusion, the first generative test in the suite, arrived in the v4.1 app release of February 11, 2025. Mostafa El-Khamy, co-chair of the MLPerf Mobile working group, had set out the plan in May 2024. Looking ahead, the MLPerf Mobile group is working to add generative AI tasks to a future version of the benchmark suite. Results come from chip and device makers that run the app and submit their logs. The published v6.0 results hold one submission in the closed division, from Samsung on an Exynos 2600 phone, and it covers the vision tests, language understanding, super resolution and Stable Diffusion rather than the new Llama ones. Samsung’s logs give a 90th percentile single stream latency of 0.56 ms for image classification, 0.83 ms for MobileBERT and 1.87 s for one Stable Diffusion image. The v5.0 round carried submissions from Samsung and Qualcomm Innovation Center. Screenshots: Samsung's submission to MLPerf Mobile v6.0, published by MLCommons. Submissions are held back until review closes under the MLCommons submission rules, which keep every result confidential inside a review committee after the deadline and let submitters file objections that the committee then decides. MLCommons publishes the accepted runs in the results table on the MLPerf Inference Mobile page, with the logs and calibration notes in a per-round repository on GitHub. The app is available through Google Play, the Apple App Store and the MLPerf Mobile repository under the Apache 2.0 license. Source: https://mlcommons.org/2026/06/mlperf-mobile-v6/ Read the article: https://llmobile.news/ticker/mlperf-inference-mobile/BenchmarksAndroidNPUhttps://llmobile.news/ticker/on-device-diffusion-llm-npu/https://llmobile.news/ticker/on-device-diffusion-llm-npu/Thu, 11 Jun 2026 14:44:00 +0200Tuowei Wang, Yanfan Sun and Ju Ren of Tsinghua University and Beihang University published llada.cpp on June 11, 2026, an inference framework that runs diffusion language models on a phone’s NPU. On an OnePlus Ace5 Pro with a Snapdragon 8 Elite, the authors measure LLaDA-8B-Instruct generating 128 tokens 38 times faster than the same model on the phone’s CPU for GSM8K prompts and 42 times faster on BoolQ, and they report a range of 17 times to 42 times across their phones and tasks. The same figure puts Llama-3-8B-Instruct on the NPU at 19 to 21 times that CPU baseline, and the authors state that llada.cpp runs LLaDA-8B up to 3.9 times faster than an autoregressive model of the same size. Chart: Wang, Sun and Ren, CC BY 4.0. A diffusion language model does not emit one token after another. It starts from a block of masked positions, 32 tokens by default here, and refines all of them together over repeated denoising steps, committing the positions it is most confident about and carrying the rest into the next step. That turns decoding into a run of large matrix operations, which is the work a mobile NPU is built for, while autoregressive decoding hands the NPU one token’s worth of work per step. The cost the paper identifies is that every denoising step recomputes the full sequence, so the parallel work gets repeated many times over. llada.cpp is built on llama.cpp and targets Qualcomm’s Hexagon NPU with 4-bit weights. Once a block is nearly done only a few masked tokens are left, so the framework pulls tokens from the next block into the same NPU pass to keep it full, keeps already committed tokens open to revision and repairs the unstable ones on the CPU while the NPU carries on, and repacks the limited address space the NPU can see so weights and cache do not have to be remapped between steps. The authors’ breakdown of a 128-token GSM8K request on the Snapdragon 8 Elite puts it at 2996.2 s on the CPU, 607.0 s once the prefix KV cache is reused, and 16.1 s with the full framework. Accuracy moves in both directions. Measured on 200-sample subsets against LLaDA-8B-Instruct on the CPU, the authors’ table has GSM8K rising from 39.0 to 43.5 and ARC-C from 84.0 to 85.0, while BoolQ falls from 82.5 to 80.5 and HellaSwag from 51.0 to 49.5, and their ablation shows the borrowed future-block tokens costing accuracy on all four tasks before the CPU-side revision path recovers it. The work covers three 16 GB phones, an OnePlus 12 with a Snapdragon 8 Gen 3, the Ace5 Pro, and an OnePlus 15 with a Snapdragon 8 Elite Gen 5, and the authors state that on the Snapdragon 8 Gen 3 their framework is still slower than Llama-3-8B at 128 tokens and only pulls ahead on the two newer chips. The paper is on arXiv under CC BY 4.0 and names no code release. Source: https://arxiv.org/abs/2606.13740 Read the article: https://llmobile.news/ticker/on-device-diffusion-llm-npu/NPUAndroidllama.cppResearchhttps://llmobile.news/ticker/caped-privacy-defence/https://llmobile.news/ticker/caped-privacy-defence/Wed, 10 Jun 2026 22:48:00 +0200Researchers at the Chinese University of Hong Kong, Southeast University and Shandong University published CAPED on June 10, 2026, a filter that blacks out parts of a phone’s screenshots before they reach a cloud-hosted GUI agent. Across a 28-task privacy suite the authors built themselves, weighted seeded leakage falls from 0.766 with raw screenshots to 0.268 with the filter in front, counted only on runs where the agent still finished its task. A multimodal model plays the attacker and scores what it can still infer from the whole screenshot sequence, not from single frames. A screenshot-based agent operates a phone the way a person does, by looking at the screen, so every image it uploads carries whatever happens to sit next to the thing the user asked for. The authors call this incidental visual privacy exposure and name contacts, message previews, non-target photos, filenames, recommendation feeds and health widgets as content the remote model receives although the user never mentioned it. In one of their test shops, a request to buy sunglasses puts prenatal vitamins, a baby bottle set and a maternity dress in front of the model, because the shop’s recommendation rows share the page with the product. Stripping names and numbers out of the text does not close that gap, and the authors measure text-only anonymisation at 0.752 against 0.766 for no protection at all. Figure: Shen et al., licensed CC BY 4.0. The mock SunShop task asks the agent to buy sunglasses, so the personalised rows around it count as incidental exposure. CAPED decides per interface element rather than per screen, according to the paper. It first sorts the current screen into one of three postures, so galleries, maps, files and financial views show their text but hide their images, while messaging, contacts, feeds and health dashboards hide both by default. The user’s instruction is turned into a short list of what the agent is allowed to need, and a hidden element is released only when it matches that list, checked either by text matching or by asking a vision model whether a cropped thumbnail really shows the requested subject. Hidden areas keep their place in the layout under a solid overlay with an eye icon, and names in exposed text are replaced by placeholders the remote agent can still follow. The defence is meant to run on the phone, and the authors argue the instruction has to be interpreted locally too, since it can name people, relationships, locations and intentions before any screen policy applies. The prototype they measured does not run on a phone. They state that CAPED’s components ran on the evaluation host with an evaluation-side service for the heavier models, and that the timings should be read as live-loop prototype latency rather than physical-phone latency, energy or thermal behaviour. Over 1,394 measured steps the privacy work added 2.325 seconds per agent step, 22.6 percent of a 10.835-second step, most of it the detector at 1.811 seconds, plus 5.716 seconds once per task to extract the requirements. The protection also costs task success. On AndroidWorld’s 116 ordinary tasks, GUI-Owl-7B completed 77 of them (66.4%) unprotected and 64 (55.2%) behind the full filter, which the authors call a prototype-level utility cost. They name further limits themselves, writing that screens their rule base does not cover fall back to the most permissive posture, which they describe as not privacy-conservative, and that open-ended requests such as finding the latest receipt widen exposure and account for much of the leakage that remains. On the connected Gmail, Drive and Photos tasks the filter moves leakage from 0.402 to 0.347 only, the interactive unmasking path was never tested with real users, and the authors write that they plan to release the CAPED artifact in a future public version. Source: https://arxiv.org/abs/2606.12666 Read the article: https://llmobile.news/ticker/caped-privacy-defence/AgentsAndroidPrivacyResearchBenchmarks