News / #model-release Tag Model releases 500 articles archived under #model-release · RSS Sign in to follow r/LocalLLaMA community 3d ago Qwen3.8-Flash-Next Strata pi beat Claude-code Opus 5.5 on a 5090 EDIT to address that the title is misleading: Qwen3.8-Flash-Next Strata pi beat Claude-code Opus 5.5 on a 5090 on a benchmark dedicated to compare model for everyday tasks. not to discriminate frontier model. --- Original post: I'm using Qwen3.8-Flash-Next on Strata with pi as… 16 r/LocalLLaMA community 3d ago [Model] Support MiniCPM-V 4.7 by tc-mb · Pull Request #29416 · ggml-org/llama.cpp Now all we need is a new model. Let me remind you that MiniCPM-V-4.7-35B-A3B was spotted on r/LocalLLaMA a few days ago (but the model was later hidden on HF).   submitted by   /u/jacek2023 [link]   [comments] 17 r/LocalLLaMA community 4d ago Benefits of using bigger models than Qwen 3.8 flash next? I recently started using local LLM for mostly coding tasks for my personal projects. I also have found local models responding much better for topics such as health, fitness, ergonomics etc. compared to the quality I had observed with ChatGPT Go (which is still a basic level)… 8 r/LocalLLaMA community 4d ago Goodbye Coding Plans I’d say we’re a year-ish away from local + good harness being enough to write 90% of code. Looking at you, Qwen. Edit: damn, sounds like you all think Qwen3.8 Next Flash is there already… going to have to give it a spin…   submitted by   /u/Purple-Programmer-7 [link]… 34 r/LocalLLaMA community 4d ago Engineer / developer observations of Gemma4-31B, Qwen3.8-27B, and 6.1-Sol for software engineering work For what it's worth. I'm a mechanical engineer by education and software engineer in practice over the past ~20 years. Playing around with local models has been a recent hobby. Today I decided to do a few tests on things that I'd consider analogous to "real" work that I do,… 38 r/LocalLLaMA community 4d ago Is anyone running Qwen3.8 Flash Next with a 1M context? So, while doing a search to read the model card again, and because I didn't memorize the huggingface URL, I saw an AI "answer" at the top of the search, stating that while it natively supports a ~244k context, it could go to 1M using YaRN. That's the first I have heared of both,… 5 r/LocalLLaMA community 4d ago 48Gb VRAM speed AND quality ! (Qwen 3.8 27B Swift 1.5 W8A16) Hi, Because sometimes you need both speed AND quality, I made my own Qwen 3.8 27B Swift 1.5 quant. It's an 8 bits AWQ + GPTQ quant. Made with love and precision (read the model card) on my 4 RTX 3090, I believe it's the best compromise you can actually get on 48Gb of VRAM.… 16 r/LocalLLaMA community 4d ago Qwen3.8-27B on a single 3090: 140 tok/s on code with a custom megakernel I've been using Claude Opus 5.5 to speed up Qwen3.8-27B on my PC (rtx 3090), it wrote a CUDA megakernel that is 1.4-1.9x faster than llama.cpp depending on the task/context length. The results and code are below: Results (same hardware, my megakernel vs llama.cpp with MTP): -… 16 r/LocalLLaMA community 4d ago Fully local conversational AI: Whisper + Hermes 8B + Kokoro, zero cloud, running inside a plush toy A conversational AI plush toy where nothing leaves my network. Everything runs on my LAN: STT: Whisper LLM: Hermes 3 Llama 3.1 8B (Q8_0) TTS: Kokoro Memory: per-person (LangGraph). It knows who it's talking to and remembers past conversations Vision: face identity (dlib) +… 18 r/LocalLLaMA community 4d ago Strata with Qwen3.8 Flash Next UD-Q4_K_XL Been seeing quite a few posts about Strata lately, so I figured I'd give it a shot on my RTX PRO 6000 and I am very impressed. Most of the benchmarks I've seen are using IQ2 or IQ3 quants, so I wanted to see how Unsloth's UD-Q4_K_XL performs instead. Setup: GPU: 1x RTX PRO 6000… 35 r/LocalLLaMA community 4d ago Sharing my latest project: strata-mlx Strata is impressive: it runs Qwen3.8-Flash-Next, a 125B-parameter mixture-of-experts model, on a 12 GB graphics card. But its authors have said macOS is out of scope. I still wanted it on the Mac. So I spent a day and a half and wrote strata-mlx. It is an unofficial MLX engine,… 35 r/MachineLearning community 4d ago Help With Choosing Hardware [P] ​ I am going to fine-tune a satire model, with the base model being `Qwen3.5-14B-Base`. The dataset has ~18000 examples, most of which are pretty long and do not align with the model's internal knowledge (they have incorrect answers to facts), so I needed to do a full… 25 r/LocalLLaMA community 4d ago big or small? what size do you want? tell them on X: https://x.com/QwenDevs/status/2108764909798641737   submitted by   /u/jacek2023 [link]   [comments] 25 r/LocalLLaMA community 4d ago Created laya : Now Introducing a new 800 Million Param physics-based typed decision model with 73k context and image support Around 3 weeks ago, I posted on this subreddit about how Jev used my one-year-old architecture, og post: https://www.reddit.com/r/LocalLLaMA/s/VpuMJt577S Then, just after that, I introduced Laya, the first open-source version of JEV, and it has reached a very large scale, thanks… 16 Latent.Space news-outlet 4d ago [AINews] TypeSafe/Jev at >$100M ARR, $7.5B valuation 3 weeks after launch Wow. 22 r/LocalLLaMA community 4d ago Qwen 3.6 35B A3B: 131K context + vision on 6GB VRAM Running HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive Q4_K_M with llama.cpp at ~600 tok/s prefill and 23 tok/s decode, 131k context window, Q8 KV cache - on an RTX 2060 6GB + 32GB DDR4 RAM. Speeds start at ~600 tok/s prefill / 23 tok/s decode on an empty KV cache. As… 21 r/LocalLLaMA community 4d ago Been a while since I posted about my harness OpenLumara.. I only post when a huge new feature releases. I think this is one of them! This is my new meditation module for my harness OpenLumara . It started out as a demonstration of just how powerful its new webui extensions system is, but has grown to be quite the meditation app in its own right. As you will hear in the video, it's a full audiovisual… 29 r/LocalLLaMA community 4d ago 3.66B formal logic model built on Granite 4.2 3B · Hugging Face webAI released TwIL LM3 Pro on September 30. I haven't seen it posted here yet, so I went through the model card, and their comparison chart is attached. It's a 3.66B model built on IBM's Granite 4.2 3B and tuned only for formal logic. That means things like checking whether a… 5 r/LocalLLaMA community 4d ago I have an ESC4000 G3 with 8x T4s in it - what is the fastest way I can deploy Qwen3.5-9B for about 10-15 users concurrently: currently using llama.cpp Hi All: I have an Asus ESC 4000 G3 with 128 GB DDR4 RAM - I tried putting in my V620s but couldn’t put more than 2. Sadly, pivoted to T4s, these are 72 watt passive cards and I thought I could use them like how I use the Mi50 32GB - but I was very wrong. It has no support from… 18 r/LocalLLaMA community 4d ago Basalt: Flash-Next at 665 tok/s structured, 354 prose on a 5090 + 5060 Ti (2.6x Strata) Basalt: Blackwell inference engine Hi all! So over the last week I've been working on a Strata fork that's heavily tuned and can achieve throughput up to 2.6x what Strata usually does on the same weights. It's designed as a specialized engine that only supports Qwen3.8… 7 r/LocalLLaMA community 4d ago Qwen Flash Next @ 137 tok/s & 3,497 tok/s Prefill w/ 512k context on a 5090, 192gb ram, Windows Build, comparing Strata and Infernix Strata vs Infernix on the same model, RTX 5090 — A/B at 262k and 512k context Hi folks, another inference engine post for your feed. I was blown away by Strata, but saw someone else post about Infernix with some wild claims, and it doesn't seem so well known, so I've spent a day… 19 r/LocalLLaMA community 4d ago What does LocalLLaMA think of REA? https://github.com/morluto/rea hit #1 a couple of days ago on github   submitted by   /u/Sufficient-Scar4172 [link]   [comments] 37 r/LocalLLaMA community 4d ago My "Anthropic/ChatGPT in a box" is... now open source and publicly hosted on Github. It's been a long time coming, and I should have done this in the first place. But LumaBrowser is now open source. Since I first released it a year ago, it's had several thousand installs and an absolutely tremendous amount of feedback. And it's come a long way. I don't call this… 17 r/LocalLLaMA community 4d ago Qwen3.8-27B UD-IQ4_XS Heretic + MTP on a 16 GB card with 55-68 tok/s (24gb and 12gb versions available too) I wanted the uncensored Qwen3.8-27B (llmfan46's Heretic build, MTP head preserved) on my RTX 4080 with MTP on (because I've been testing TONS of configs, and realized that would be the one) And none of the published quants were built for that: the good IQ4_XS doesn't leave room… 4 Simon Willison community 4d ago Deno is joining Cloudflare Deno is joining Cloudflare The Deno team released the first version of celld back in August - their open source implementation of the Durable Objects pattern from Cloudflare Workers. Today, Cloudflare are acquiring Deno outright, with the goal of building on celld to "make… 7 TechCrunch — AI news-outlet 4d ago The maker of non-text AI model Jev valued at $7.5B just weeks after launch TypeSafe AI raised $870 million in a round led by a16Z. 24 r/LocalLLaMA community 4d ago Qwen3.8 27b with 200K ctx + MTP on 12GB Ampere Cards Hello all! Thank you to everyone that contributed their feedback and notes for LlamAmpere the last time I posted. I've continued to chip away at improvements for the 3090 crowd -- this release is modestly faster (3-4%), with a few hundred MB smaller runtime (if you're using… 5 r/LocalLLaMA community 4d ago Is there a music embedding model? Has anybody built/released a music embedding model trained on a large library of diverse music?   submitted by   /u/unraveleverything [link]   [comments] 29 r/LocalLLaMA community 5d ago H2O-Lightning-4B: Apache-2.0 4B Decision model, official #1 open model on JevBench (above Jev) Disclosure: I work at H2O.ai. We released H2O-Lightning-4B, an open-weight (Apache-2.0) model for the "decisions API" style of inference that Jev made popular: you send a state plus typed questions (pick one / yes-no / score), and get calibrated probabilities back from a single… 6 llama.cpp releases dev-tools 5d ago b11535 llama-bench: respect -fitc if bigger than required benchmark size ( #28331 ) Assisted-By: opencode,llama.cpp,Qwen3.6-35B-A3B,Qwen3.8-27B Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54430110 macOS/iOS: macOS Apple Silicon (arm64)… 30 r/LocalLLaMA community 5d ago Just another purely open-weight models benchmark Live results, coding benchmarks included, agentic benchmarks included, domains and other classifications filters included. (Spoiler: DeepSeek-V4-Vision-Exp rules, but other models have their rule areas): https://beta.locallm.top Evaluated by domain experts (my friends mostly;… 14 r/LocalLLaMA community 5d ago Qwen 3.8 Flash Next-GSQ-RCO-IQ2_XS at ~21 tok/s on just an RTX 3060 12GB + 16GB DDR4 RAM(No gate pruning, 100% bit-exact) (Don't judge by the screenshot, the cache is cold. It hits 24+ tok/s with a warm cache!) About two months ago, I made a post here asking whether predicting which MoE experts would be used on the next token could actually help speed up CPU/GPU offloading. Original post: Tried… 11 r/LocalLLaMA community 5d ago GLM 5.3 Flash opensource @ the top of Artificial Analysis Cyber Index over Claude. It seems Dario's "too powerful for you users" strategy is paying off: we have two open models at the top of the leaderboard, surpassing every single model from Anthropic. Open source prevails. Even Mistral Large 4 is better!   submitted by   /u/LegacyRemaster [link]… 8 r/LocalLLaMA community 5d ago GitHub - google-ai-edge/ml-drift: GPU-Accelerated AI/ML Inference Blog Post : ML Drift: Next-Gen GPU AI/ML Inference at the Edge - Google Developers Blog The Google AI Edge Team is excited to announce the open-source release of ML Drift , our high-performance, cross-platform, on-device GPU compute engine specifically built for on-device AI/ML… 29 llama.cpp releases dev-tools 5d ago b11532: model : use exact GELU for ModernBERT encoders (#30108) model : use exact GELU for ModernBERT encoders Assisted-by: Codex model : keep tanh GELU aliases on ggml_geglu Assisted-by: Claude Opus 5.5 model : map gelu_python to ggml_geglu_erf Assisted-by: Claude Opus 5.5 18 Hacker News — AI on Front Page community 5d ago Python 3.15 Article URL: https://www.python.org/downloads/release/python-3150/ Comments URL: https://news.ycombinator.com/item?id=50021127 Points: 213 # Comments: 40 4 r/LocalLLaMA community 5d ago Qwen3.8-Flash-Next-GSQ-RCO (IQ3_S): ~20-30 tok/sec decode & 300-90k tok/sec prefill on 12GB VRAM + 32GB RAM + NVME I got Qwen3.8-Flash-Next-GSQ-RCO-Abliterated running at IQ3_S with just 12GB VRAM and 32GB system RAM, achieving 20-30 tok/sec decode (Q2 achieves 39-45 tok/s) & 300 to ~90 thousand tok/sec prefill @ 131k context, on a custom fork of Strata. This fork has tonnes of architectural… 4 r/LocalLLaMA community 5d ago Qwen/Qwen-Image-2.1-Turbo · Hugging Face   submitted by   /u/chocofoxy [link]   [comments] 29 r/LocalLLaMA community 5d ago Qwen-Image-2.1-Turbo released! Qwen-Image-2.1-Turbo, create and edit images in just 8 denoising steps! Open weights now available! Built on Qwen-Image-2.1, Turbo is an accelerated checkpoint on the same 7B visual generation architecture. Fewer steps does not mean lower quality: it still generates strong 2K… 38 ElevenLabs official-blog 5d ago Introducing Brand Kit Add your logo, fonts and colors once, and everything you make pulls them in automatically. Ask for a launch campaign for a new product and it comes back on-brand. 8 Hacker News — AI on Front Page community 5d ago Nobel Peace Prize for 2026 to Navanethem "NAVI" Pillay Article URL: https://www.nobelprize.org/prizes/peace/2026/press-release/ Comments URL: https://news.ycombinator.com/item?id=50018420 Points: 217 # Comments: 126 13 r/LocalLLaMA community 5d ago Stepping away from Benchmarks and Code, what models are you using for Creating writing projects All anyone ever talks in this sub is about x new model having x benchmark scores and how well it can code. Or some new tech to increase t/s for qwen. So I wanted to go back to the roots of this sub and talk about how well local models can write. I am not going to mention closed… 35 OpenAI official-blog 5d ago Asana cuts model costs 76x in browser tests with GPT-6.1 Sol Using GPT-6 Astra in Codex, Asana made its browser agent 76x cheaper and 5x faster in tests to offer customers more capable models. 16 r/LocalLLaMA community 5d ago [Paper] EngramEdit: Decoupled Knowledge Updates in LLMs through Conditional Memory Conditional memory architectures such as DeepSeek Engram use input n-grams to look up learned embeddings, expanding the capacity of large language models (LLMs) with limited additional computation. Beyond model scaling, this architecture has demonstrated the potential to… 15 r/LocalLLaMA community 5d ago Running the uncensored Qwen3.8-27B (HauhauCS) on a 4090 at 262K context and ~130 tok/s HauhauCS ships their uncensored Qwen3.8-27B as GGUF only. NInfer, a C++/CUDA wanted its own format. Now the same model that ran at 91.6 tok/s / 131K under llama.cpp does: 262K context (the model's full native window) ~130 tok/s decode with MTP3, 70.8% acceptance 3,591 tok/s… 5 arXiv — Machine Learning research 5d ago Ranking Prior Alignment for Credit Risk Modeling: When Do External Priors Matter? arXiv:2610.11146v1 Announce Type: new Abstract: Cold-start credit scoring -- deploying models with scarce labeled data, weak features, or minimal capacity -- is a recurring problem in financial machine learning. When a new lending product launches, labeled default data is… 21 arXiv — Machine Learning research 5d ago RideBench: A Large-Scale Exogenous-Aware Benchmark for Ride-Hailing Time Series Forecasting arXiv:2610.11164v1 Announce Type: new Abstract: We release Ride-Hailing, a large-scale ride-hailing time series dataset synthesized from DiDi's marketplace data across 200 spatial areas. Ride-Hailing spans four consecutive years at half-hourly granularity and covers three… 36 arXiv — Machine Learning research 5d ago Do Flatter Minima Drive Better Generalization? An Algorithmic Separation in Grokking arXiv:2610.11206v1 Announce Type: new Abstract: Flat loss landscapes have long been linked to better generalization in neural networks. However, its role as a causal mechanism for generalization is less established. Grokking provides an unique testbed to understand this… 8 arXiv — Machine Learning research 5d ago MC-TRCM: Observation-Aware Recursive Fusion for Incomplete Mobile and Wearable Mental-Health Feature Views arXiv:2610.11408v1 Announce Type: new Abstract: Public mobile and wearable mental-health datasets often provide summarized feature tables rather than synchronized raw sensor streams. In these releases, each anchor corresponds to a survey or label time and may combine phone or… 34 arXiv — NLP / Computation & Language research 5d ago SWE-Journey: Towards More Realistic Evaluation of Coding Assistants through Long-Horizon, Multi-Turn Interaction arXiv:2610.11559v1 Announce Type: new Abstract: Coding assistants such as Claude Code and Codex have become a major application of LLM agents, yet existing benchmarks remain far from real-world use, particularly in task horizon and interaction length. Code assistants require… 35 Page 3 of 10 · 500 articles ← Newer Older →
