News / #edge Tag Edge 407 articles archived under #edge · RSS Sign in to follow arXiv — Machine Learning research 2d ago A Layer Importance Metric for Quantization Accounting for the Speed-Quality Trade-off in Autoregressive Models arXiv:2608.26926v1 Announce Type: new Abstract: Small language models (sLLMs) are nowadays hosted on devices with limited memory and computational budget. In an autoregressive setup, inference is memory-bandwidth bound: uniform quantization is often detrimental to such models,… 11 r/LocalLLaMA community 2d ago Ornith-1.5-35B-A3B on 8 GB VRAM: I think I've found my sweet spot A few days ago I posted asking what people considered the best local model for an 8 GB VRAM GPU . At the time, my personal sweet spot was Qwen3.6-35B-A3B , for agentic coding with Pi.dev. Well… Thanks to the suggestions in that thread, I think I've found something even better.… 25 r/LocalLLaMA community 2d ago No, Engrams won't let you run 1T models locally. It does something even better. Ever since Qwen 3.8 Flash Next dropped, there's a misconception going around that N-gram tables will let people run 1T+ parameter models on a single server with 980B parameters offloaded to SSD. I'm here to disappoint you: it won't. But what it will actually do for local models… 31 r/LocalLLaMA community 2d ago I used local Qwen 27b to build a harness and replace OpenCode Sharing my harness for running local LLMs that I built using Qwen 3.x 27B (> 90% locally built) under my supervision - not vibe-coded. Its free, no telemetry, and open-source (AGPL). Works on Windows, Linux (sorry, no Mac yet). I use it for my own coding + mixed workflows. How… 13 r/LocalLLaMA community 3d ago Local LLM harness for reverse engineering software? Does anyone know of a decent reverse engineering harness/workspace setup for local models that I can just point the model at and have it go to work until it's reversed most if not all of the functions in a binary, even if it takes days? Of course, I'm willing to put effort into… 13 Ars Technica — AI news-outlet 3d ago IBM's new Granite 4.2 models ride the wave of interest in local LLMs The focus is on agentic capability and predictable enterprise deployment. 30 arXiv — Machine Learning research 4d ago Renormalization Group Flow Matching for Scalable Local Generative Modeling arXiv:2608.23696v1 Announce Type: new Abstract: Despite their remarkable success in modeling complex data, generative models face a fundamental tradeoff. Global approaches can capture full structural coherence but suffer from high computational costs, while local models are… 26 Hugging Face Daily Papers research 5d ago MobilePA-Bench: Benchmarking Mobile Planner Agents on Complex Real-World Tasks Abstract MobilePA-Bench is an interactive sandbox benchmark that evaluates mobile planning agents on tool-calling, sub-agent collaboration, memory usage, and composite skill invocation under real runtime constraints. Generated by thinkingmachines/Inkling-Small As on-device LLM… 13 r/LocalLLaMA community 5d ago Please join r/LowEndLocalAI, a community for running local LLMs on low spec hardware If you’re trying to run local LLMs on a normal laptop, an older desktop, integrated graphics, limited VRAM, or simply the hardware you already own, r/LowEndLocalAI is meant for you. The idea is simple: What useful things can we do with the hardware we already have? I’ve been… 24 r/LocalLLaMA community 5d ago MobileMoE - a facebook Collection MobileMoE is a family of on-device Mixture-of-Experts (MoE) language models with sub-billion active parameters, designed to push the quality–efficiency Pareto frontier for on-device LLMs, including three model scales (S/M/L): 0.3B/0.5B/0.9B active parameters (1.3B/2.8B/5.3B… 7 r/LocalLLaMA community 5d ago What's the best local model you've found for 8 GB of VRAM? I'm curious what other people are using for local LLM coding / agentic coding with only 8 GB of VRAM . My current setup is: Intel Core i7-11800H RTX 3070 Laptop , 8 GB VRAM 32 GB DDR4 RAM openSUSE Tumbleweed / KDE Unsloth Studio pi.dev as the coding agent After testing quite a… 33 arXiv — Machine Learning research 6d ago Thermo-FL: Thermal-Aware Robust Federated Fine-Tuning of Large Language Models for Edge AI arXiv:2608.21172v1 Announce Type: new Abstract: Federated fine-tuning enables large language models to adapt on edge devices without centralizing private data, but practical deployments must address hardware instability and adversarial update corruption together. Thermally… 4 r/LocalLLaMA community 6d ago Qwen 3.8 27B for actual local programming Most YouTube benchmarks only show trivial tasks like generating landing pages or simple Three.js games. Is a local model like Qwen 3.8 27B actually capable of real-world systems programming—such as building GTK4 or Qt 6 applications in Rust or C++ with external libraries?… 10 Hacker News — AI on Front Page community 7d ago Why your local LLM feels dumber than it is Article URL: https://forum.level1techs.com/t/why-your-local-llm-feels-dumber-than-it-is/253917 Comments URL: https://news.ycombinator.com/item?id=49402232 Points: 201 # Comments: 69 24 r/LocalLLaMA community 7d ago Current best model for narrative, chat, prompt creation (so basically everything except agentic coding)? - 5090 Im looking to set up a new local llm (probably on unsloth studio as that seemed to be doing pretty well last time I tested it). This one won't need to do agentic coding or app building or anything (not this time) but instead more 'text' based tasks such as - being given… 10 r/LocalLLaMA community 7d ago Fixed the MTP head on Ornith1.5 35B A3B. +3% TPS -33% wall clock I love the Ornith 35B local models, 1.0 has been running my HAM radio rig for me. I have a hackRF receiver and a 5 watt quansheng portable the both run headless through the PC. I tried out the new Ornith1.5 build and it was faster and more accurate than 1.0. I read the threads… 30 r/LocalLLaMA community 7d ago This is a great sub, regardless of what complaints people have about it. This is a genuine community of real generally respectful adult human beings. Despite the enthusiasm all of you have for local AI, you can recognize that there are times when local LLMs are flawed, and even how practical they are to use for the majority of people to use. Go over… 12 r/LocalLLaMA community 8d ago How to give a local LLM/agent access to a "real" web browser I can't seem to find a good answer to this, my Hermes agent has access to Firecrawl and some other web scrapers for content extraction, but anyone know of a way to let a local LLM drive a "real" web browser? My wife asked me to have Hermes go and look at her LinkedIn profile,… 21 r/LocalLLaMA community 9d ago Ornith-1.5-35B-A3B-NInfer - 250 tok/s, 5-8k prefill, 5090 I tried this model yesterday, and it felt to me like the best one I've tried for a local model for interactive use; the responses and reasoning are very fast, and it actually performs agentic tasks well. The speed is phenomenal. I am running this on Ninfer for Windows -… 37 Hacker News — AI on Front Page community 9d ago Show HN: I trained a 125M model to autocomplete piano on-device I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model… 38 r/LocalLLaMA community 10d ago TinySearch v0.6.1 - still a lightweight web research tool for local LLMs, now with bring-your-own-browser support Hey everyone, Posted TinySearch here a few versions ago and got a bunch of useful feedback, so figured I'd post an update because the thing has changed quite a bit since then. Repo: [ https://github.com/TinySuiteHQ/TinySearch]() The basic idea is still the same: TinySearch is a… 30 r/LocalLLaMA community 10d ago Qwen3.8-27b has the highest level of "agency" I've ever seen in a local model Off a single prompt, given my credentials and the name of my university, qwen3.8-27b was able to successfully pull my class schedule from the kinda shitty and convoluted web of university websites. It needed no human intervention, and executed 80 tool calls. Another time, I… 28 r/LocalLLaMA community 10d ago Anyone NOT on full auto when coding with local LLMs? Would love to know who's letting a 9B just go ham locally, haha But in all seriousness, how many of you are keeping to manual or manual-ish dev workflows?   submitted by   /u/BatPlack [link]   [comments] 23 NVIDIA Developer Blog official-blog 10d ago Post-Train NVIDIA Cosmos 3 Edge for On-Device Robot Control Robots need policies that can adapt to their sensors, environments, and tasks while running on onboard computing hardware. World models offer a foundation for... 24 r/LocalLLaMA community 11d ago Am I doing something wrong? Qwen 3.8 27B seems useless for agentic coding I have been using local models on/off for like 2 years or so but never really used them extensively because the closed ones were always much better. Once Qwen 3.8 27B was released I decided to give it another serious try. I configured Cline and ZooCode as VSCode addons,… 29 r/LocalLLaMA community 11d ago local models fear my tests am I the only one who does this lol   submitted by   /u/close_Meal6005 [link]   [comments] 20 arXiv — NLP / Computation & Language research 12d ago Wiola 13M, a Gated Spiral Attention Architecture for Parameter Efficient Small Language Models arXiv:2608.14604v1 Announce Type: new Abstract: Small language models in the ten to one hundred million parameter range are attractive for on device inference, rapid experimentation, and controlled scientific study, yet most of them reuse the standard transformer block without… 14 Hugging Face Daily Papers research 13d ago MobileMem: Learning from a Year of Mobile Experiences Abstract MobileMem is a benchmark and framework for evaluating on-device long-term memory through year-scale, multimodal mobile experience trajectories that require temporal reasoning, knowledge updating, and preference inference. Generated by thinkingmachines/Inkling-Small The… 5 r/LocalLLaMA community 13d ago Simon Willison: Qwen 3.8 27B is excellent, but it defaults to wildly overthinking things I look to Simon for a broad survey of current LLM tech. Here's his review of playing with Qwen 3.8 27B . His comment on Mastodon was "I can't remember the last time I've had this much fun playing with a local model that runs on my own computers". BTW, the "wildly overthinking"… 30 r/LocalLLaMA community 14d ago SOTA Apple Silicon Inference (August 15, 2026) This is a HANDWRITTEN post. I spent way too much time trying to get fast inference on Apple Silicon. This post is for people who want to know what's the latest on running local models on their mac, and why they may not be seeing the performance others in the community claim.… 34 r/LocalLLaMA community 14d ago club-5060ti refresh: tested RTX 5060 Ti presets, a proper high-context harness, and Qwen3.8 27B Quick update on the RTX 5060 Ti local LLM repo. It has changed quite a bit since my previous posts. The project started as a collection of practical notes and benchmark results. That was useful, but as the dataset grew it became harder to answer the question most people actually… 10 r/LocalLLaMA community 14d ago If you would have told me half a year ago that a local model running in my office would be able to one-shot a Super Mario clone, I would have called you nuts. Qwen3.8-27B is a different beast. Running the Q8 GGUF on my Framework Desktop is not fast, but it's extremely smart for overnight batches and background jobs. Can't wait to play around with MTP and other quants. Have any of you found ways to improve speed while keeping accuracy?… 7 r/LocalLLaMA community 15d ago (Newbie) what do you use local models for? I’m a long time lurker and have a newbie rig. i run Qwen 3.6 27B and I use LM studio. I use it to summarize long and terse financial and legal documents that I don’t want to upload to cloud. that isn’t everyday though and I am not really learning anything new. id like for this… 32 r/MachineLearning community 15d ago If you had a bunch of GPUs lying around, what would you actually build with them? (Running LLMs is off the table) [D] Be honest if someone dropped a stack of high-end GPUs on your desk tomorrow, what would you actually do with them? And before the usual answers roll in: running local LLMs is banned for this thread. It’s been done to death and feels pretty pointless at this point. So… what else?… 28 Hugging Face Daily Papers research 17d ago Ready Cohorts: Bounding GPU Opportunity and Avoiding Host Round Trips in LLM-Agent Control Abstract Two studies define measurable GPU control gates for LLM-agent services by analyzing concurrent cohort scheduling and on-device routing versus host redispatch. Generated by thinkingmachines/Inkling-Small LLM-agent services repeatedly execute small deterministic… 25 r/LocalLLaMA community 17d ago LiquidAI/LFM2.5-VL-3B · Hugging Face LFM2.5-VL-3B is a multimodal variant of LFM2.5, a family of hybrid models designed for on-device deployment . It builds on LFM2-VL-3B with further mid- and post-training. LFM2.5-VL-3B can process both text and images, and uses the LFM2.5-2.6B language model as its backbone,… 15 r/LocalLLaMA community 18d ago RAG for regular users? One of the reasons I got into local LLMs was the possibility of getting answers using my own documents and books (a few hundreds) instead of having to search through them manually. However since I'm not a data specialist or an engineer, RAG projects were too difficult for me,… 17 r/LocalLLaMA community 18d ago Anthropic, OpenAI, Google, Meta, Microsoft, and Mistral all signed the EU Code of Practice on Transparency of AI-Generated Content Even open source local models from these companies will be watermarking code and text since it's required by law.   submitted by   /u/Bestlife73 [link]   [comments] 19 r/LocalLLaMA community 19d ago What can us 8 GB VRAM poors do? I want to hook up a local model to Cline, but it seems the best model is still just Qwen 3.5 9B. Please can we have a Qwen 3.8 9B that gets close to Qwen 3.6 27B?   submitted by   /u/Aggravating-Push-207 [link]   [comments] 31 Hugging Face Daily Papers research 19d ago Ego-OSCAR: Egocentric Open source Stereo CAptuRe System Abstract We present Ego-OSCAR, an open-hardware, low-cost, head-mounted stereo-inertial capture device for egocentric data collection in the wild. EgoOSCAR pairs a hardware-synchronized global-shutter stereo camera with a 6- axis IMU, an embedded Linux SBC for on-device video… 28 r/LocalLLaMA community 19d ago I made a web-design benchmark for local models (Muse Glimmer 30B vs Qwen 3.6 27b vs Deepseek V4 Flash 0731)   submitted by   /u/ShadyShroomz [link]   [comments] 34 r/LocalLLaMA community 19d ago Please Share Your Experience About Muse Glimmer I have a classic test for local LLM's. I asked for 8 ball pool game with only one HTML file and Muse Glimmer spend 21k Token(I m using full context so 128k) and only created a 220 lines of HTML and said its done. With my experience its not even close to Qwen 3.6 27B and we are… 19 r/LocalLLaMA community 19d ago Best open-source harness like Claude Code? Avid claude code user here looking to do equivalent things with local models. Just want to plug in something like Qwen and have the interface be 1:1 with claude code. Any suggestion?   submitted by   /u/Neighbor_ [link]   [comments] 36 r/LocalLLaMA community 19d ago Best Local LLMs - August 2026 Wowee!! Just when you thought it couldn't get better for open weight models, we probably have had our best period yet!?!?! Models that rival the closed frontier, Opus level models on non-insane hardware and more. A massive industry alliance coming out in support of open AI in… 25 r/LocalLLaMA community 20d ago Meta open sources new on-device model Muse Glimmer & Muse spark 1.2 also coming soon!   submitted by   /u/provoloner09 [link]   [comments] 17 r/LocalLLaMA community 20d ago I've added Maple-Preview to Mference, got 40 tps generation with 500MB of used RAM on Air M4 I like the idea of running local models, but I don’t like the idea of having them eat up all of my memory. I’ve always thought that the best way to build an edge model would be to make something smart enough to reason over data, but without requiring much knowledge of its own.… 14 arXiv — Machine Learning research 20d ago MAUPITI: On-Device Prototype-Based Learning on a Smart Infrared Sensor arXiv:2608.07192v1 Announce Type: new Abstract: Low-resolution infrared (IR) array sensors represent an interesting solution for privacy-preserving human sensing in embedded systems. In this letter, we describe a smart multi-pixel IR sensor integrating a 16$\times$16 thermal… 13 r/LocalLLaMA community 20d ago small model for text-to-regex Anyone have a good local model for this? i feel like this should be a near solved problem.   submitted by   /u/minaminotenmangu [link]   [comments] 23 r/LocalLLaMA community 21d ago Building a budget 32GB → 48GB VRAM home AI server: 2-3x RX 9060 XT 16GB vs RTX 5060 Ti 16GB, AM5 vs used EPYC? I’m planning a dedicated home AI server, mainly for local LLM inference, agents/tool use, Docker services, and eventually larger MoE models with CPU offload. My plan is to start with 2x 16GB GPUs = 32GB VRAM , but I want to build the platform from day one knowing that I’ll… 37 r/MachineLearning community 22d ago Built a tool to generate slides from research papers using local LLMs (because I hate formatting decks and privacy matters) [P] Hi guys, Every time I had to prepare a presentation based on a paper or research doc, I found the process super tedious. Plus, I really dislike uploading unpublished stuff or sensitive data to online AI services just to get a draft. So I put together a tool called… 22 Page 1 of 9 · 407 articles Older →
