Tag

Edge

500 articles archived under #edge · RSS

Google DeepMind official-blog 19h ago

Google DeepMind and Singapore Government announce new AI tool to protect citizens against scams

Learn how Google DeepMind and GovTech Singapore uses lightweight on-device Gemma models to intercept conversational scams in private chats with zero cloud data egress.

15
r/LocalLLaMA community 22h ago

Claude CoWork for Local LLMs that runs on your Phone

Hi Friends, I am a solo developer. Ever since I had Qwen 3.8 running on my RTX4090, I had an itch to use it on my phone for everything. That gave me the idea to build SteerClaw Mobile - a complete agentic experience on the Phone where the entire agent loop runs on it. We just…

35 1
r/LocalLLaMA community 1d ago

LiquidAI/LFM2.5-24B-A2B

Gguf's: https://huggingface.co/LiquidAI/LFM2.5-24B-A2B-GGUF Free Demo: https://playground.liquid.ai "LFM2.5-24B-A2B is a 24B-parameter reasoning model with only 2B active parameters, built for fast, private agents on-device . It is the largest and final model in the LFM2.5…

24
Hugging Face official-blog 1d ago

Run fast, private agents on-device with LFM2.5-24B-A2B

Back to Articles a]:hidden"> Run fast, private agents on-device with LFM2.5-24B-A2B Team Article Published October 13, 2026 Upvote 7 Raahul Vignesh Raahul26 LiquidAI Song Duong sduong LiquidAI Akhilesh Deepak Gotmare akhileshdeepakgotmare LiquidAI Fernando Fernandes Neto…

15
Hacker News — AI on Front Page community 1d ago

A Dynamic and Distributed Virtual GPU Cloud Built on Idle Apple Silicon Devices

Article URL: https://machinelearning.apple.com/research/distributed-gpu-cloud Comments URL: https://news.ycombinator.com/item?id=50070593 Points: 214 # Comments: 90

9
r/LocalLLaMA community 1d ago

SonicBoom: 52 tok/s decode on a laptop RTX 3050 (llama.cpp: 45) with the same quantized kernels, the difference is a graph compiler. No Python.

I’ve been building SonicBoom, a native tensor runtime written in C++/CUDA with a C ABI and no Python anywhere in the stack. I just tagged v0.0.1, and the first target is local LLM inference on consumer GPUs. Numbers (Gemma 4 E4B-it Q3_K_M on a laptop RTX 3050 8 GB, both runtimes…

12
r/LocalLLaMA community 1d ago

How far are we from a local model being able to optimize it's own runtime stack?

A lot of projects have come up recently consisting of specific hardware config specialized forks of existing conferencing engines, but they all seem to be made using Claude or OpenAI's biggest frontier models. So given that open source is about 4-6 months behind, can we…

32
r/LocalLLaMA community 2d ago

LLC: lightweight Open WebUI alt now with Agents+MCP+RAG+Image Editing

Quick context: LocalLightChat (LLC) is a chat frontend for local LLMs. Download it, run it, that's it. No installation (unless you want to), no dependencies, and it runs on pretty much anything, including ancient hardware. I originally built it because Open WebUI kept feeling…

23
r/LocalLLaMA community 2d ago

Any takers for local llm? NVIDIA’s CUDA Moat Is Accelerating RTX Spark Laptop Sales, Higher 128GB Unified Memory Options Belonging To Surface Laptop Ultra Already Out Of Stock

Cheapest one I see at MC is the asus pro-art version for 6499. https://www.microcenter.com/product/715662/asus-proart-p14-14-wqxga-120hz-oled-laptop   submitted by   /u/chemist_slime [link]   [comments]

33
r/MachineLearning community 2d ago

I built a tiny table that learns your local LLM's classification answers and serves the repeats in microseconds [D]

project: https://2796gaurav.github.io/shad0w/ If you use a local model for classification, you know the wait: seconds per message, for questions it has answered hundreds of times. shad0w sits in front of any OpenAI-compatible server (Ollama works out of the box:…

14
r/LocalLLaMA community 2d ago

What are you doing for a reliable, well designed local model based AI search?

Ive been using OpenWebUI's inbuilt web search and I can't put my finger on it, but it's not quite as polished, reliable, seamless as using Claude or Gemini. And its often slow beyond a limit that makes me subconsciously not want to use it - I understand its unlikely to get to…

11
r/LocalLLaMA community 3d ago

I built a lightweight tool that automatically frees up VRAM for gaming and restores your local LLMs afterward

I run local LLMs pretty much around the clock, but I also enjoy gaming on the same PC. Having models sitting in VRAM while gaming isn't exactly ideal, and constantly unloading and reloading them manually got annoying. So I built GamePause . It's a lightweight, open-source…

33
r/LocalLLaMA community 3d ago

Reverse Engineering w/ Local?

Two questions? Can a local model like Qwen 3.8 27B reverse engineer games and programs? If not, is there a fine tuned model that can?   submitted by   /u/Danny_Davitoe [link]   [comments]

29
r/LocalLLaMA community 4d ago

Benefits of using bigger models than Qwen 3.8 flash next?

I recently started using local LLM for mostly coding tasks for my personal projects. I also have found local models responding much better for topics such as health, fitness, ergonomics etc. compared to the quality I had observed with ChatGPT Go (which is still a basic level)…

8
r/LocalLLaMA community 4d ago

Engineer / developer observations of Gemma4-31B, Qwen3.8-27B, and 6.1-Sol for software engineering work

For what it's worth. I'm a mechanical engineer by education and software engineer in practice over the past ~20 years. Playing around with local models has been a recent hobby. Today I decided to do a few tests on things that I'd consider analogous to "real" work that I do,…

38
r/LocalLLaMA community 4d ago

Is there a better option than llama.cpp for 4GB VRAM for Higher tokens/sec?

I love the idea of running local models on consumer hardware. I currently use `llama.cpp`, but I’ve been looking for a "better" alternative for a while now. I want something that utilizes my system resources more efficiently—for instance, by managing my RTX with 4GB VRAM more…

32
r/LocalLLaMA community 4d ago

microsoft doubles down on local ai with nvidia, but with a cost

https://preview.redd.it/uyjw2lcrvkuh1.png?width=1536&format=png&auto=webp&s=ce991be22faec3de5cbaf39119c14782b6ed0af3 link to article TLDR; new microsoft surface and nvidia rtx spark laptops are starting at 2.6k and hitting nearly 7k for the 128gb unified memory models. local llm…

5
r/LocalLLaMA community 5d ago

LlamaTale v0.43.0 - MCP Server for story creation

In the dawn of the LLM craze, when wild Llama2's roamed free and 4k tokens was considered a fairly long context for local models, I forked an interactive fiction and MUD library called Tale to experiment with LLM generated content. The idea being a text based adventure with…

23
Ollama releases dev-tools 5d ago

v0.40.3-rc0

server: skip local model compatibility migration ( #18908 )

7
Ollama releases dev-tools 5d ago

v0.40.3

server: skip local model compatibility migration ( #18908 )

21
r/LocalLLaMA community 5d ago

GitHub - google-ai-edge/ml-drift: GPU-Accelerated AI/ML Inference

Blog Post : ML Drift: Next-Gen GPU AI/ML Inference at the Edge - Google Developers Blog The Google AI Edge Team is excited to announce the open-source release of ML Drift , our high-performance, cross-platform, on-device GPU compute engine specifically built for on-device AI/ML…

29
r/LocalLLaMA community 5d ago

Stepping away from Benchmarks and Code, what models are you using for Creating writing projects

All anyone ever talks in this sub is about x new model having x benchmark scores and how well it can code. Or some new tech to increase t/s for qwen. So I wanted to go back to the roots of this sub and talk about how well local models can write. I am not going to mention closed…

35
r/LocalLLaMA community 5d ago

Qwen3.8-27B: 159 tok/s on R9700, 64 tok/s on Strix Halo

LemonSeed Studio is an iPad editor/IDE with on-device inference on an AMD GPU in a Thunderbolt enclosure. It embeds the unmodified upstream Linux amdgpu + amdkfd driver (mac_linuxgpu) as a PCIDriverKit extension, and runs LemonSeed Engine (LSE) on the GPU. In the photos: iPad…

36
r/LocalLLaMA community 6d ago

chatgpt's new intelligent ui was reverse engineered in less than 24 hours, and apparently you can recreate it with local llms

came across a pretty interesting technical breakdown of chatgpt's newly launched "intelligent ui" feature, and thought this subreddit might find it interesting. for anyone unfamiliar with the concept, intelligent ui is essentially openai's take on generative ui . instead of…

17
TechCrunch — AI news-outlet 6d ago

Google releases a new local-first Granola competitor

Google’s new AI Edge Foresight app takes on Granola with an offline meeting note-taker that can transcribe conversations, generate notes, and answer questions using on-device AI.

11
arXiv — NLP / Computation & Language research 6d ago

How Fragile Is On-Device Language Model Safety? Localizing Safety-Critical Parameters for Sparse Fault Analysis

arXiv:2610.09000v1 Announce Type: cross Abstract: As small language models (SLMs) are increasingly deployed on resource-constrained and on-device platforms, including as components of agentic systems, the integrity of locally stored model parameters becomes an important safety…

19
The Information — AI news-outlet 7d ago

Microsoft Debuts New Windows PCs, Features Powered by On-Device AI

Microsoft on Wednesday unveiled updates to Windows software and hardware that will allow Windows PCs to run more AI models locally, which the company said would bring down the cost of using Copilot and other AI agents for customers. The new announcements are part of Microsoft’s…

10
r/LocalLLaMA community 7d ago

Local Qwen 3.8 27B vs DeepSeek Flash API: Is local good enough?

Running a model on your own machine used to be a privacy story with a quality tax. On this generation that trade has narrowed to where we can state it plainly: for daily work, the local model is good enough. We measured it — 25 paired tasks across four workloads, same prompts,…

36
Google DeepMind official-blog 8d ago

EmbeddingGemma 2: an open, lightweight multimodal embedding model

EmbeddingGemma 2: an open, lightweight multimodal embedding model Oct 06, 2026 | x.com Facebook LinkedIn Mail EmbeddingGemma 2 is the most capable model for on-device multimodal embeddings, natively mapping combinations of text, images, audio, and video into a unified embedding…

4
Ollama releases dev-tools 8d ago

v0.40.0

What's Changed Models run on MLX on Apple Silicon by default In this release, on Apple Silicon devices, model architectures supported by the MLX runtime will automatically run on MLX. ollama pull qwen3.8 ollama run qwen3.8 Additional models include gemma4 , qwen3.6 and qwen3.5…

9
r/LocalLLaMA community 8d ago

I made a Chrome extension that filters your YouTube feed with a small local model running in the browser (WebGPU, no server)

My YouTube feed was mostly songs, pranks and celebrity clips, so I built a filter that judges each video title with a small decision model running inside Chrome. Nothing is sent anywhere: no server, no API key, no account, and after one download it works offline. What it does -…

37
Ollama releases dev-tools 8d ago

v0.40.0

What's Changed Models run on MLX on Apple Silicon by default In this release, on Apple Silicon devices, model architectures supported by the MLX runtime will automatically run on MLX. ollama pull qwen3.8 ollama run qwen3.8 Additional models include gemma4 , qwen3.6 and qwen3.5…

23
r/LocalLLaMA community 8d ago

Qwen 3.8 27b just feels… ok?

I’ve seen posts here raving about how good Qwen 3.8 27b is. The benchmarks look incredible, and all the online discourse seems to deem it the best local model. I have 32GB VRAM and run Unsloth’s Q6 version with OpenCode. For small tasks, it feels fine. I range 30-40 t/s decode…

15
r/LocalLLaMA community 8d ago

PewDiePie getting banned twice by OpenAI while making a local model is top-tier comedy 💀

So PewDiePie decides to fine-tune a local AI model called Ajax on his own computer. Pretty normal stuff for local model fans. To make his dataset, he uses OpenAI's API. OpenAI catches him using their outputs to train another model, flags his account for breaking their terms, and…

30
Ollama releases dev-tools 9d ago

v0.40.0

What's Changed Models run on MLX on Apple Silicon by default In this release, on Apple Silicon devices, model architectures supported by the MLX runtime will automatically run on MLX. ollama pull qwen3.8 ollama run qwen3.8 Additional models include gemma4 , qwen3.6 and qwen3.5…

11
r/LocalLLaMA community 9d ago

Is Strix Halo (GMKtec EVO-X2, etc.) the closest thing we have to a "dream" local LLM box?

I've been looking at the <32B model space and keep coming back to an interesting question. A few years ago, projects like Hummingbird+ suggested that cheap custom accelerators (FPGA-based) might become the future of local inference. But today it seems like memory capacity is…

15
r/LocalLLaMA community 10d ago

Built a quick, sub-15ms Rust CLI/TUI to pack repos into prompts without burning 40k tokens on lockfiles and junk

Whenever I feed codebases into local models (Qwen, DeepSeek R1) or API models, the biggest annoyance is prompt pollution: - Lockfiles (`Cargo.lock`, `package-lock.json`) burning 30,000+ tokens for zero reason. - SVGs, binary files, or build artifacts slipping into the context. -…

32
r/LocalLLaMA community 10d ago

How long before we have a local model capable of modeling?

I'm using Qwen 3.8 and qwen flash on a 5090. It's miles behind the latest Opus 5.5. Even if it was remotely capable it would be a huge help to me, but for now, with regards to 3D modeling, local models are not close at all.   submitted by   /u/Hot_Masterpiece_3668 [link]…

35
r/LocalLLaMA community 10d ago

Is there any way to improve creative writing for Local Models (Qwen)?

I wanted to know if theres anything that can improve creative writing for our Local Models? I specificly am asking for qwen models as they are way better when it comes to researching and writing html files compared to gemma / muse (which I know are better at creative writing).…

35
r/MachineLearning community 10d ago

Top ARC-ΑGI-3 scores on Kaggle just went from 7% to 56% [N]

https://preview.redd.it/gkjgii48gfth1.png?width=575&format=png&auto=webp&s=4e1c35bdacf41d18a6409baedcddc9432ec79814 This happened over the past 30 days. So smallish local models (Kagglers can only use those), in a harness, just started beating average humans at a benchmark…

25
r/LocalLLaMA community 10d ago

The curse of 64GB system RAM

Not a bot. Not a Strata shill. Just sharing my experience. So, I have an R9700 in my machine, plus an RTX 5060 Ti, and 64GB DDR5 system RAM. Overall, not a bad setup. Anyway, I mainly run a daily driver local LLM on the R9700 while running image/video inference on ComfyUI on the…

32
r/LocalLLaMA community 10d ago

Local Web Search Safety

Hi all, How you guys handling safe deployment of websearch in Hermes, pi and other harnesses? Does anyone have a good uproars setup guide for local models? I tried to implement a sandboxed search system but it caused endless tool calls. Want to guard against prompt injection and…

15
r/LocalLLaMA community 11d ago

Anyworld, a self-hosted multiplayer text RPG where a local LLM is the Dungeon Master

Hey everyone, I’ve been working on a game called Anyworld. It’s a browser-based multiplayer (single player also supported) text adventure inspired by the early days of AI Dungeon, especially its browser-based free version AI Dungeon 2. The setup is pretty straightforward: one…

28
r/LocalLLaMA community 11d ago

Fixed long-horizon task drift on local setups using a deterministic state plugin

Ran into an annoying issue with local models on long tasks. Once context window compaction hits after a few thousand tokens, the model loses sight of the original scope. Even with good system prompts, a few compaction cycles cause goal drift, hallucinated task completion, or…

10
r/LocalLLaMA community 12d ago

I made my own cybersecurity benchmark and ran Qwen3.8 27B, here's how a local model actually does at hacking

Hey local AI community, I've been working on this for a while and finally feel ok sharing it. It's a cyber benchmark where the model gets a shell in an isolated docker box and has to find the exact flag. Pwn, web, crypto, rev, forensics, a few real CVEs and some multi-stage…

4
r/LocalLLaMA community 12d ago

best <40B alternatives to Qwen/Deepseek for (1) Coding (2) Long document QA test

Due to some reasons, Qwen/Deepseek Chinese models are NOT allowed in the my workplace. So, what local models, do you think, is the best alternatives to Qwen/Deepseek for (1) Coding (2) Long document QA test (like giving a long medical history of 120k tokens and ask a question…

23
r/LocalLLaMA community 12d ago

Detecting hallucinations in local models without eating VRAM: What we learned testing 1.5B to 120B models

Hey everyone, If you run local models via Ollama in production or personal projects, you've probably run into the hallucination problem: how do you know when a model is hallucinating without burning extra VRAM or waiting 5 seconds for a heavy judge model? The standard academic…

8
r/LocalLLaMA community 12d ago

I "built" a local LLM radio like the Pharaohs built the pyramids

My local stack is a couple DGX Sparks, a 5090, and a 4070 TI. This radio site took three local LLMs, and we generate music, voices, and wacky sound effects. I give you https://pilgrim.farm It's a lot more farm than pilgrim.   submitted by   /u/jwhh91 [link]  …

25
r/LocalLLaMA community 13d ago

szmcp: a ZIM HTML to Markdown converter and yet another ZIM MCP server

Hello all, I wanted to share a little project I vibe-coded for myself that you may find useful. As many people here like to suggest, I wanted to give my small local LLMs access to information to improve their world knowledge. I didn't want to give my LLM free reign searching and…

24
r/LocalLLaMA community 13d ago

Best local model for Blender and game dev?

I have been looking at some local models, but even the smartest ones like GLM5.3 Flash and DSv4 Flash have a hard time creating coherent models in Blender and placing them logically in game engines. Is this something that local models are just too dumb still to do good job at?…

19

Google DeepMind and Singapore Government announce new AI tool to protect citizens against scams

Claude CoWork for Local LLMs that runs on your Phone

LiquidAI/LFM2.5-24B-A2B

Run fast, private agents on-device with LFM2.5-24B-A2B

A Dynamic and Distributed Virtual GPU Cloud Built on Idle Apple Silicon Devices

SonicBoom: 52 tok/s decode on a laptop RTX 3050 (llama.cpp: 45) with the same quantized kernels, the difference is a graph compiler. No Python.

How far are we from a local model being able to optimize it's own runtime stack?

LLC: lightweight Open WebUI alt now with Agents+MCP+RAG+Image Editing

Any takers for local llm? NVIDIA’s CUDA Moat Is Accelerating RTX Spark Laptop Sales, Higher 128GB Unified Memory Options Belonging To Surface Laptop Ultra Already Out Of Stock

I built a tiny table that learns your local LLM's classification answers and serves the repeats in microseconds [D]

What are you doing for a reliable, well designed local model based AI search?

I built a lightweight tool that automatically frees up VRAM for gaming and restores your local LLMs afterward

Reverse Engineering w/ Local?

Benefits of using bigger models than Qwen 3.8 flash next?

Engineer / developer observations of Gemma4-31B, Qwen3.8-27B, and 6.1-Sol for software engineering work

Is there a better option than llama.cpp for 4GB VRAM for Higher tokens/sec?

microsoft doubles down on local ai with nvidia, but with a cost

LlamaTale v0.43.0 - MCP Server for story creation

v0.40.3-rc0

v0.40.3

GitHub - google-ai-edge/ml-drift: GPU-Accelerated AI/ML Inference

Stepping away from Benchmarks and Code, what models are you using for Creating writing projects

Qwen3.8-27B: 159 tok/s on R9700, 64 tok/s on Strix Halo

chatgpt's new intelligent ui was reverse engineered in less than 24 hours, and apparently you can recreate it with local llms

Google releases a new local-first Granola competitor

How Fragile Is On-Device Language Model Safety? Localizing Safety-Critical Parameters for Sparse Fault Analysis

Microsoft Debuts New Windows PCs, Features Powered by On-Device AI

Local Qwen 3.8 27B vs DeepSeek Flash API: Is local good enough?

EmbeddingGemma 2: an open, lightweight multimodal embedding model

v0.40.0

I made a Chrome extension that filters your YouTube feed with a small local model running in the browser (WebGPU, no server)

v0.40.0

Qwen 3.8 27b just feels… ok?

PewDiePie getting banned twice by OpenAI while making a local model is top-tier comedy 💀

v0.40.0

Is Strix Halo (GMKtec EVO-X2, etc.) the closest thing we have to a "dream" local LLM box?

Built a quick, sub-15ms Rust CLI/TUI to pack repos into prompts without burning 40k tokens on lockfiles and junk

How long before we have a local model capable of modeling?

Is there any way to improve creative writing for Local Models (Qwen)?

Top ARC-ΑGI-3 scores on Kaggle just went from 7% to 56% [N]

The curse of 64GB system RAM

Local Web Search Safety

Anyworld, a self-hosted multiplayer text RPG where a local LLM is the Dungeon Master

Fixed long-horizon task drift on local setups using a deterministic state plugin

I made my own cybersecurity benchmark and ran Qwen3.8 27B, here's how a local model actually does at hacking

best <40B alternatives to Qwen/Deepseek for (1) Coding (2) Long document QA test

Detecting hallucinations in local models without eating VRAM: What we learned testing 1.5B to 120B models

I "built" a local LLM radio like the Pharaohs built the pyramids

szmcp: a ZIM HTML to Markdown converter and yet another ZIM MCP server

Best local model for Blender and game dev?