News / #edge Tag Edge 500 articles archived under #edge · RSS Sign in to follow r/LocalLLaMA community 1mo ago Sliding-window beats linear attention Interesting new paper from Alexia Jolicoeur-Martineau (of Tiny Recursive Model fame) and collaborators. They seem to be able to replace quadratic attention with sliding window attention + attention sinks and no post training. This could be big for memory constrained local LLM… 25 r/LocalLLaMA community 1mo ago The Chrono Trigger plot challenge - Crono awakens in his modest bedroom of 2095... A hallucinated event, Crono awakens in his modest bedroom of 2095... I am using local models since 2025 January. My daily driver is qwen 3.6 35B A3B, which works pretty well for coding tasks, but I always benchmark the models for lexical knowledge as well, where the models… 38 r/LocalLLaMA community 1mo ago I fine-tuned a 0.8B local model for dictation cleanup. It matched a hosted frontier model on this narrow task I built SpeakoFlow Mini, an Apache-2.0 fine-tune of Qwen3.5-0.8B for dictation cleanup. It is not a chat model or a general rewriter. It takes speech-to-text output, applies corrections the speaker actually made, and leaves everything else alone. That last part is harder than it… 21 arXiv — Machine Learning research 1mo ago A Layer Importance Metric for Quantization Accounting for the Speed-Quality Trade-off in Autoregressive Models arXiv:2608.26926v1 Announce Type: new Abstract: Small language models (sLLMs) are nowadays hosted on devices with limited memory and computational budget. In an autoregressive setup, inference is memory-bandwidth bound: uniform quantization is often detrimental to such models,… 11 r/LocalLLaMA community 1mo ago Ornith-1.5-35B-A3B on 8 GB VRAM: I think I've found my sweet spot A few days ago I posted asking what people considered the best local model for an 8 GB VRAM GPU . At the time, my personal sweet spot was Qwen3.6-35B-A3B , for agentic coding with Pi.dev. Well… Thanks to the suggestions in that thread, I think I've found something even better.… 25 r/LocalLLaMA community 1mo ago No, Engrams won't let you run 1T models locally. It does something even better. Ever since Qwen 3.8 Flash Next dropped, there's a misconception going around that N-gram tables will let people run 1T+ parameter models on a single server with 980B parameters offloaded to SSD. I'm here to disappoint you: it won't. But what it will actually do for local models… 31 r/LocalLLaMA community 1mo ago I used local Qwen 27b to build a harness and replace OpenCode Sharing my harness for running local LLMs that I built using Qwen 3.x 27B (> 90% locally built) under my supervision - not vibe-coded. Its free, no telemetry, and open-source (AGPL). Works on Windows, Linux (sorry, no Mac yet). I use it for my own coding + mixed workflows. How… 13 r/LocalLLaMA community 1mo ago Local LLM harness for reverse engineering software? Does anyone know of a decent reverse engineering harness/workspace setup for local models that I can just point the model at and have it go to work until it's reversed most if not all of the functions in a binary, even if it takes days? Of course, I'm willing to put effort into… 13 Ars Technica — AI news-outlet 1mo ago IBM's new Granite 4.2 models ride the wave of interest in local LLMs The focus is on agentic capability and predictable enterprise deployment. 30 arXiv — Machine Learning research 1mo ago Renormalization Group Flow Matching for Scalable Local Generative Modeling arXiv:2608.23696v1 Announce Type: new Abstract: Despite their remarkable success in modeling complex data, generative models face a fundamental tradeoff. Global approaches can capture full structural coherence but suffer from high computational costs, while local models are… 26 Hugging Face Daily Papers research 1mo ago MobilePA-Bench: Benchmarking Mobile Planner Agents on Complex Real-World Tasks Abstract MobilePA-Bench is an interactive sandbox benchmark that evaluates mobile planning agents on tool-calling, sub-agent collaboration, memory usage, and composite skill invocation under real runtime constraints. Generated by thinkingmachines/Inkling-Small As on-device LLM… 13 r/LocalLLaMA community 1mo ago Please join r/LowEndLocalAI, a community for running local LLMs on low spec hardware If you’re trying to run local LLMs on a normal laptop, an older desktop, integrated graphics, limited VRAM, or simply the hardware you already own, r/LowEndLocalAI is meant for you. The idea is simple: What useful things can we do with the hardware we already have? I’ve been… 24 r/LocalLLaMA community 1mo ago MobileMoE - a facebook Collection MobileMoE is a family of on-device Mixture-of-Experts (MoE) language models with sub-billion active parameters, designed to push the quality–efficiency Pareto frontier for on-device LLMs, including three model scales (S/M/L): 0.3B/0.5B/0.9B active parameters (1.3B/2.8B/5.3B… 7 r/LocalLLaMA community 1mo ago What's the best local model you've found for 8 GB of VRAM? I'm curious what other people are using for local LLM coding / agentic coding with only 8 GB of VRAM . My current setup is: Intel Core i7-11800H RTX 3070 Laptop , 8 GB VRAM 32 GB DDR4 RAM openSUSE Tumbleweed / KDE Unsloth Studio pi.dev as the coding agent After testing quite a… 33 arXiv — Machine Learning research 1mo ago Thermo-FL: Thermal-Aware Robust Federated Fine-Tuning of Large Language Models for Edge AI arXiv:2608.21172v1 Announce Type: new Abstract: Federated fine-tuning enables large language models to adapt on edge devices without centralizing private data, but practical deployments must address hardware instability and adversarial update corruption together. Thermally… 4 r/LocalLLaMA community 1mo ago Qwen 3.8 27B for actual local programming Most YouTube benchmarks only show trivial tasks like generating landing pages or simple Three.js games. Is a local model like Qwen 3.8 27B actually capable of real-world systems programming—such as building GTK4 or Qt 6 applications in Rust or C++ with external libraries?… 10 Hacker News — AI on Front Page community 1mo ago Why your local LLM feels dumber than it is Article URL: https://forum.level1techs.com/t/why-your-local-llm-feels-dumber-than-it-is/253917 Comments URL: https://news.ycombinator.com/item?id=49402232 Points: 201 # Comments: 69 24 r/LocalLLaMA community 1mo ago Current best model for narrative, chat, prompt creation (so basically everything except agentic coding)? - 5090 Im looking to set up a new local llm (probably on unsloth studio as that seemed to be doing pretty well last time I tested it). This one won't need to do agentic coding or app building or anything (not this time) but instead more 'text' based tasks such as - being given… 10 r/LocalLLaMA community 1mo ago Fixed the MTP head on Ornith1.5 35B A3B. +3% TPS -33% wall clock I love the Ornith 35B local models, 1.0 has been running my HAM radio rig for me. I have a hackRF receiver and a 5 watt quansheng portable the both run headless through the PC. I tried out the new Ornith1.5 build and it was faster and more accurate than 1.0. I read the threads… 30 r/LocalLLaMA community 1mo ago This is a great sub, regardless of what complaints people have about it. This is a genuine community of real generally respectful adult human beings. Despite the enthusiasm all of you have for local AI, you can recognize that there are times when local LLMs are flawed, and even how practical they are to use for the majority of people to use. Go over… 12 r/LocalLLaMA community 1mo ago How to give a local LLM/agent access to a "real" web browser I can't seem to find a good answer to this, my Hermes agent has access to Firecrawl and some other web scrapers for content extraction, but anyone know of a way to let a local LLM drive a "real" web browser? My wife asked me to have Hermes go and look at her LinkedIn profile,… 21 r/LocalLLaMA community 1mo ago Ornith-1.5-35B-A3B-NInfer - 250 tok/s, 5-8k prefill, 5090 I tried this model yesterday, and it felt to me like the best one I've tried for a local model for interactive use; the responses and reasoning are very fast, and it actually performs agentic tasks well. The speed is phenomenal. I am running this on Ninfer for Windows -… 37 Hacker News — AI on Front Page community 1mo ago Show HN: I trained a 125M model to autocomplete piano on-device I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model… 38 r/LocalLLaMA community 1mo ago TinySearch v0.6.1 - still a lightweight web research tool for local LLMs, now with bring-your-own-browser support Hey everyone, Posted TinySearch here a few versions ago and got a bunch of useful feedback, so figured I'd post an update because the thing has changed quite a bit since then. Repo: [ https://github.com/TinySuiteHQ/TinySearch]() The basic idea is still the same: TinySearch is a… 30 r/LocalLLaMA community 1mo ago Qwen3.8-27b has the highest level of "agency" I've ever seen in a local model Off a single prompt, given my credentials and the name of my university, qwen3.8-27b was able to successfully pull my class schedule from the kinda shitty and convoluted web of university websites. It needed no human intervention, and executed 80 tool calls. Another time, I… 28 r/LocalLLaMA community 1mo ago Anyone NOT on full auto when coding with local LLMs? Would love to know who's letting a 9B just go ham locally, haha But in all seriousness, how many of you are keeping to manual or manual-ish dev workflows?   submitted by   /u/BatPlack [link]   [comments] 23 NVIDIA Developer Blog official-blog 1mo ago Post-Train NVIDIA Cosmos 3 Edge for On-Device Robot Control Robots need policies that can adapt to their sensors, environments, and tasks while running on onboard computing hardware. World models offer a foundation for... 24 r/LocalLLaMA community 1mo ago Am I doing something wrong? Qwen 3.8 27B seems useless for agentic coding I have been using local models on/off for like 2 years or so but never really used them extensively because the closed ones were always much better. Once Qwen 3.8 27B was released I decided to give it another serious try. I configured Cline and ZooCode as VSCode addons,… 29 r/LocalLLaMA community 1mo ago local models fear my tests am I the only one who does this lol   submitted by   /u/close_Meal6005 [link]   [comments] 20 arXiv — NLP / Computation & Language research 1mo ago Wiola 13M, a Gated Spiral Attention Architecture for Parameter Efficient Small Language Models arXiv:2608.14604v1 Announce Type: new Abstract: Small language models in the ten to one hundred million parameter range are attractive for on device inference, rapid experimentation, and controlled scientific study, yet most of them reuse the standard transformer block without… 14 Hugging Face Daily Papers research 1mo ago MobileMem: Learning from a Year of Mobile Experiences Abstract MobileMem is a benchmark and framework for evaluating on-device long-term memory through year-scale, multimodal mobile experience trajectories that require temporal reasoning, knowledge updating, and preference inference. Generated by thinkingmachines/Inkling-Small The… 5 r/LocalLLaMA community 1mo ago Simon Willison: Qwen 3.8 27B is excellent, but it defaults to wildly overthinking things I look to Simon for a broad survey of current LLM tech. Here's his review of playing with Qwen 3.8 27B . His comment on Mastodon was "I can't remember the last time I've had this much fun playing with a local model that runs on my own computers". BTW, the "wildly overthinking"… 30 r/LocalLLaMA community 1mo ago SOTA Apple Silicon Inference (August 15, 2026) This is a HANDWRITTEN post. I spent way too much time trying to get fast inference on Apple Silicon. This post is for people who want to know what's the latest on running local models on their mac, and why they may not be seeing the performance others in the community claim.… 34 r/LocalLLaMA community 2mo ago club-5060ti refresh: tested RTX 5060 Ti presets, a proper high-context harness, and Qwen3.8 27B Quick update on the RTX 5060 Ti local LLM repo. It has changed quite a bit since my previous posts. The project started as a collection of practical notes and benchmark results. That was useful, but as the dataset grew it became harder to answer the question most people actually… 10 r/LocalLLaMA community 2mo ago If you would have told me half a year ago that a local model running in my office would be able to one-shot a Super Mario clone, I would have called you nuts. Qwen3.8-27B is a different beast. Running the Q8 GGUF on my Framework Desktop is not fast, but it's extremely smart for overnight batches and background jobs. Can't wait to play around with MTP and other quants. Have any of you found ways to improve speed while keeping accuracy?… 7 r/LocalLLaMA community 2mo ago (Newbie) what do you use local models for? I’m a long time lurker and have a newbie rig. i run Qwen 3.6 27B and I use LM studio. I use it to summarize long and terse financial and legal documents that I don’t want to upload to cloud. that isn’t everyday though and I am not really learning anything new. id like for this… 32 r/MachineLearning community 2mo ago If you had a bunch of GPUs lying around, what would you actually build with them? (Running LLMs is off the table) [D] Be honest if someone dropped a stack of high-end GPUs on your desk tomorrow, what would you actually do with them? And before the usual answers roll in: running local LLMs is banned for this thread. It’s been done to death and feels pretty pointless at this point. So… what else?… 28 Hugging Face Daily Papers research 2mo ago Ready Cohorts: Bounding GPU Opportunity and Avoiding Host Round Trips in LLM-Agent Control Abstract Two studies define measurable GPU control gates for LLM-agent services by analyzing concurrent cohort scheduling and on-device routing versus host redispatch. Generated by thinkingmachines/Inkling-Small LLM-agent services repeatedly execute small deterministic… 25 r/LocalLLaMA community 2mo ago LiquidAI/LFM2.5-VL-3B · Hugging Face LFM2.5-VL-3B is a multimodal variant of LFM2.5, a family of hybrid models designed for on-device deployment . It builds on LFM2-VL-3B with further mid- and post-training. LFM2.5-VL-3B can process both text and images, and uses the LFM2.5-2.6B language model as its backbone,… 15 r/LocalLLaMA community 2mo ago RAG for regular users? One of the reasons I got into local LLMs was the possibility of getting answers using my own documents and books (a few hundreds) instead of having to search through them manually. However since I'm not a data specialist or an engineer, RAG projects were too difficult for me,… 17 r/LocalLLaMA community 2mo ago Anthropic, OpenAI, Google, Meta, Microsoft, and Mistral all signed the EU Code of Practice on Transparency of AI-Generated Content Even open source local models from these companies will be watermarking code and text since it's required by law.   submitted by   /u/Bestlife73 [link]   [comments] 19 r/LocalLLaMA community 2mo ago What can us 8 GB VRAM poors do? I want to hook up a local model to Cline, but it seems the best model is still just Qwen 3.5 9B. Please can we have a Qwen 3.8 9B that gets close to Qwen 3.6 27B?   submitted by   /u/Aggravating-Push-207 [link]   [comments] 31 Hugging Face Daily Papers research 2mo ago Ego-OSCAR: Egocentric Open source Stereo CAptuRe System Abstract We present Ego-OSCAR, an open-hardware, low-cost, head-mounted stereo-inertial capture device for egocentric data collection in the wild. EgoOSCAR pairs a hardware-synchronized global-shutter stereo camera with a 6- axis IMU, an embedded Linux SBC for on-device video… 28 r/LocalLLaMA community 2mo ago I made a web-design benchmark for local models (Muse Glimmer 30B vs Qwen 3.6 27b vs Deepseek V4 Flash 0731)   submitted by   /u/ShadyShroomz [link]   [comments] 34 r/LocalLLaMA community 2mo ago Please Share Your Experience About Muse Glimmer I have a classic test for local LLM's. I asked for 8 ball pool game with only one HTML file and Muse Glimmer spend 21k Token(I m using full context so 128k) and only created a 220 lines of HTML and said its done. With my experience its not even close to Qwen 3.6 27B and we are… 19 r/LocalLLaMA community 2mo ago Best open-source harness like Claude Code? Avid claude code user here looking to do equivalent things with local models. Just want to plug in something like Qwen and have the interface be 1:1 with claude code. Any suggestion?   submitted by   /u/Neighbor_ [link]   [comments] 36 r/LocalLLaMA community 2mo ago Best Local LLMs - August 2026 Wowee!! Just when you thought it couldn't get better for open weight models, we probably have had our best period yet!?!?! Models that rival the closed frontier, Opus level models on non-insane hardware and more. A massive industry alliance coming out in support of open AI in… 25 r/LocalLLaMA community 2mo ago Meta open sources new on-device model Muse Glimmer & Muse spark 1.2 also coming soon!   submitted by   /u/provoloner09 [link]   [comments] 17 r/LocalLLaMA community 2mo ago I've added Maple-Preview to Mference, got 40 tps generation with 500MB of used RAM on Air M4 I like the idea of running local models, but I don’t like the idea of having them eat up all of my memory. I’ve always thought that the best way to build an edge model would be to make something smart enough to reason over data, but without requiring much knowledge of its own.… 14 arXiv — Machine Learning research 2mo ago MAUPITI: On-Device Prototype-Based Learning on a Smart Infrared Sensor arXiv:2608.07192v1 Announce Type: new Abstract: Low-resolution infrared (IR) array sensors represent an interesting solution for privacy-preserving human sensing in embedded systems. In this letter, we describe a smart multi-pixel IR sensor integrating a 16$\times$16 thermal… 13 Page 4 of 10 · 500 articles ← Newer Older →
