r/LocalLLaMA

500 articles archived · Visit source ↗ · RSS

r/LocalLLaMA community 12d ago

OpenCode overrides the samplers for Qwen models to the wrong values

This is invisible with llama.cpp or derivatives, but ninfer helpfully logs the sampler settings on each request and auto-configures the correct ones for the model. Basically, OpenCode will always send top-p=1.0 (which means there is no filtering of low-probability tokens except…

17
r/LocalLLaMA community 12d ago

Linux Improves VRAM Management in 7.3 Kernel 🥳

  submitted by   /u/johnnyApplePRNG [link]   [comments]

17
r/LocalLLaMA community 12d ago

Hugging Face just surpassed 3 million models on the Hub

From Hugging Face on 𝕏: https://x.com/huggingface/status/2089673018737869242?s=20   submitted by   /u/Nunki08 [link]   [comments]

30
r/LocalLLaMA community 12d ago

DeepSeek V4 Flash 0731 on Strix Halo: draft model, n_max sweep, and a launch line that actually helps

I know the front page is completely dominated by the Qwen3.8 27B. I know it's a fantastic model and everyone is rightfully discussing it, but I wanted to shift gears and talk about another model I’ve been experimenting with and got some solid results from:…

9
r/LocalLLaMA community 12d ago

Don't ignore llama.cpp RPC with old hardware. Results of a 5070 Ti and 1080 Ti over gigabit ethernet: it's actually functional.

Results up front: I had to prioritize prefill or token generation - there was no happy medium. Using UD-Q4_K_XL, q8 kv cache, and 96k max context: focus on generation (MTP = 2): 350 pp and 36 tg @ 12k context focus on prefill (disabled MTP): 560 pp and 19 tg @ 12k context Focus…

29
r/LocalLLaMA community 12d ago

Qwen 3.8 27b saved me $650+ in API costs this evening

I've been experimenting with Qwen3.8-27B using DeepSeek Harness. It's a monster at long-horizon tasks, and the results were pretty wild. DeepSeek Harness ran on my Windows PC and connected over LAN to NInfer on a separate RTX PRO 6000 box. The model was Qwen3.8-27B with a 262K…

23
r/LocalLLaMA community 12d ago

Qwen 3.8 27B is faster than expected

i ran this model on my two 5060 TI 16GB cards at Q4 in unsloth and LM studio ( i downloaded NVFP4 but didn't try it in vLLM ) i think it runs faster than expected it gives me 50 - 60 t/s with MTP. this is surprising because it's a dense model and Qwen 3.6 was giving me 30t/s…

34
r/LocalLLaMA community 12d ago

Qwen 3.8 27b vs Deepseek Flash

Hey Guys, What amazing weeks it has been for open source releases. I was really impresssed by DS flash final checkpoint and i have been playing around with it until qwen 3.8 released. I checked the benckmarks, and I dont know what to think anymore how can such a small model…

6
r/LocalLLaMA community 12d ago

tencent/UI-Mate-27B · Hugging Face

Overview UI-Mate-27B is an open-weight foundation GUI agent for long-horizon work across applications and operating systems. It observes live screenshots, reasons over the visible state, and produces structured keyboard and mouse actions for native desktop interaction. UI-Mate…

4
r/LocalLLaMA community 12d ago

Qwen 3.8 27B xhigh vs medium small comparison (+ others for fun)

It's a small experiment of mine to check thinking effort on Qwen and I do have to say xhigh does overthink but I'm not sure if it's bad because the result is rather amazing. Although the prompt was very open-ended so it took liberties. TL:DR at the bottom. Images in order: Qwen…

29
r/LocalLLaMA community 12d ago

AA is the reason for Qwen3.8 27B shipped with xhigh

I know why Qwen3.8 27B shipped with xhigh reasoning as default, it's to do its best in benchmarks. Models from top labs often get benchmarked at multiple reasoning levels, but that same treatment doesn't apply to other labs. Open models are lucky to even be benchmarked at all.…

16
r/LocalLLaMA community 12d ago

Qwen dev says not to wait for 35B-A3B

What does this mean? Is there something else coming? Maybe 122B? Or no models?   submitted by   /u/Mean-Ad1493 [link]   [comments]

13
r/LocalLLaMA community 12d ago

Optimizing Qwen3.6 / Qwen3.8-27B on 16GB VRAM: Complete Benchmark Results and Setup Guide (~30-50tps at 32k to 72k context)

This post was made with AI. I tried to remove as much slop as possible and keep it straight to the point to save your time as I know how annoying AI slop posts can be, but I still wanted to retain all the details so it can be used as a resource for comparison with other future…

38
r/LocalLLaMA community 12d ago

CDW has bumped the MSRP of the RTX Pro 6000 from $16,000 to $19,999

Did they slip up and leak future pricing? Live link: https://www.cdw.com/product/pny-nvidia-rtx-pro-6000-graphic-card-96-gb-gddr7/8326705 Archive link: https://web.archive.org/web/20260818013250/https://www.cdw.com/product/pny-nvidia-rtx-pro-6000-graphic-card-96-gb-gddr7/8326705…

19
r/LocalLLaMA community 12d ago

Made this game in two prompts with Q4, Qwen 3.8 is amazing

This took one prompt to build, and another follow up prompt to fix two issues (player got stuck with the bomb and broken enemies path-finding), this is only html, css and js, no external assets, all done by Qwen. Using UD-Q4_K_XL in llama.cpp with 128k context and k5_0/v4_1…

36
r/LocalLLaMA community 12d ago

Waiting for Qwen 3.8 35B A3B

  submitted by   /u/puffyarizona [link]   [comments]

6
r/LocalLLaMA community 12d ago

Qwen 3.8 35b and 122b - We hope/wait/beg for models incessantly. But how do we actually give the lab more incentive to make it?

So many comments begging for these models, and I get it. People from the labs frequent r/LocalLLaMA , so maybe the begging comments aren't useless. They make demand known at least. Same with polls, etc. But what could this community do to actually help the lab, or somehow give…

11
r/LocalLLaMA community 12d ago

we benchmark models nobody actually runs

qwen3.8-27b looks genuinely impressive on the benchmark tables - beating models many times its size on some of them. but those numbers come from bf16 weights, and nobody here is running a 27b at bf16. we're running the 4-bit at ~17gb because that's what fits on a 4090 or a 24gb…

16
r/LocalLLaMA community 12d ago

Qwen3.8-27B Uncensored Aggressive is out with K_P quants and HauhauCS FastMTP (up to 3.02x TG)!

The dense Qwen release is back! Qwen3.8-27B Uncensored Aggressive is out with the complete K_P quant range, Vision, native NextN, and HauhauCS FastMTP. Aggressive here means no refusals, no personality alterations, and very little preamble on difficult prompts. It keeps…

5
r/LocalLLaMA community 12d ago

Local agentic coding Benchmark : Qwen 3.8 27B (in many weights quants / cache quants / engine / reasoning effort) vs others.

In medium reasoning mode, it both scores higher than the 3.6 version, AND is very much more efficient (almost half requests needed, and a third less tokens generated) - at DeepSeek v4 Flash 3107 MXFP4 level The xhigh mode is advertised to be the best one for hard tasks. In this…

32
r/LocalLLaMA community 12d ago

Weirdly, no one talks about Temperature setting for the Qwen3.8 27b

Mind you, it is 1.0 by default, yet everyone is focused on how much the new model thinks, restricting the reasoning budget and/or dropping the reasoning level. Set the temperature to 0.7 and the model will no longer write a whole book of thoughts before trying to make a small…

8
r/LocalLLaMA community 12d ago

I pushed Qwen3.8-27B to 99 tps single request and 1150 tps with a batch request on a RTX 3090

I'm back. Yesterday I released the first version of hyper-optimized Qwen3.8-27B inference engine for a RTX 3090, reaching 82 tps on single request and 672 peak. Over the last 24 hours I've been exploring how to improve it further (without a quality degredation). Yesterday we had…

23
r/LocalLLaMA community 13d ago

"Opus 4.8 thinks too much", "Muse Glimmer sits between Gemma and Qwen, that's boring", "Gemma 4 is too lazy"

I'm starting to think there's no way to make a reasoning model that won't draw persistent vocal complaints on here. EDIT: Qwen 3.8 not Opus 4.8*, freudian slip lol   submitted by   /u/MerePotato [link]   [comments]

29
r/LocalLLaMA community 13d ago

Qwen 3.8 35bA3b wen?

Artificial analysis index scores Qwen 3.5 27b: 35 Qwen 3.6 27b: 38 Qwen 3.8 27b: 52 What the hell kind of a jump was that? Even if it is benchmaxxed, the jump is insane. Qwen3.6 35b A3b: 32 That's ~6 points behind its dense 27b model, but is ~5x faster for inference given only…

29
r/LocalLLaMA community 13d ago

llama.cpp adaptive MTP PR#27210

Just wanted to raise some attention to this PR I filed if anyone would like to try it out. This adds an adaptive MTP mode to llama.cpp which employs a fairly simple counting-style state machine to determine the appropriate depth to set the MTP depth to dynamically. The goal here…

31
r/LocalLLaMA community 13d ago

Qwen3.8 27B > Opus 5 Medium on Artificial Analysis Agentic Index

https://preview.redd.it/xh1rloaf4zjh1.png?width=1628&format=png&auto=webp&s=a536fae1b50b327f2bc55d4f4f874f94ae66e867 Thanks Qwen team!   submitted by   /u/secopsml [link]   [comments]

14
r/LocalLLaMA community 13d ago

Qwen3.8 27B = GPT-5.6 Luna compressed into 27B

How crazy is that?   submitted by   /u/kevinlch [link]   [comments]

37
r/LocalLLaMA community 13d ago

Artificial Analysis' Qwen3.8-27B benchmarks put it neck and neck with DeepSeek V4 and GPT-5.6 Luna Max

  submitted by   /u/anderspitman [link]   [comments]

30
r/LocalLLaMA community 13d ago

Why do people like coding harnesses like opencode etc instead of an IDE?

Just curious - I like to be able to see and manage the scripts my agent is working on. I find stuff like Claude Code and Open Code useful for doing stuff on my linux box but I don't understand why people would use that instead of a dedicated IDE where you can actually see the…

33
r/LocalLLaMA community 13d ago

Ling 3.0 Tiny is the strongest, fastest and greatest model on my low end PC!

This Ling 3.0 Tiny 8b param with 1.3b active is the fastest, smartest model I can run on my poor old pc, with 4gb vram. It actually runs lightning fast, like 36 token / sec, as smart as Qwen 3.5 9b / Gemma 12, (Very close), and even faster because of 1.3b active parameters. The…

17
r/LocalLLaMA community 13d ago

Qwen 3.8 27B Overthinking, It has to be done, it has to be overthinking to punch Opus 4.6

Yes, it sucks to waste time waiting on 16K+ reasoning tokens alone. But here's the thing, this is only a 27B model trying to perform on par with 1T+ parameter models. Something has to be sacrificed, and that sacrifice is the amount of reasoning or trajectory tokens. This isn't…

4
r/LocalLLaMA community 13d ago

Deepseek Harnness - why is feels better

Guys, could someone smarter than me explain what makes Deepseek Harness so efficient? I run it with local Qwen 3.8 (Q6). I tried Opencode/Openchamber (my favourite so far), Pi agent and Hermes. New Qwen seems to overthing by default but this could be minimized with some effort.…

16
r/LocalLLaMA community 13d ago

llama.cpp version v0.1.0 has been released

llama.cpp is apparently moving to semantic versioning instead of just sequential build numbers (like b10456). The first semantic version tag was created today: https://github.com/ggml-org/llama.cpp/releases/tag/v0.1.0 Congrats to llama.cpp on version v0.1.0!   submitted by…

29
r/LocalLLaMA community 13d ago

EXL3 seems to be fading from the r/LocalLLaMa consciousness, and while I suspected it, I'm surprised at this point in time.

EXL3 is an alternative to llama.cpp. And while there is extensive tooling for llama.cpp, EXL3's primary deployment ( TabbyAPI ), has a OpenAI compatible API so it shouldn't matter. Why won't this tool matter to you? If you have a GPU with under 24 GB of VRAM, the value kind of…

27
r/LocalLLaMA community 13d ago

After pushing 1M+ tokens through Qwen 3.8 27B, here is my optimal llama.cpp config for 16GB VRAM (73k Context, Agentic Coding)

Following up on my previous post about my budget server setup (Intel N100 + RTX 5060 Ti 16GB), a few of you asked for a deeper dive into my actual inference config and real-world agentic performance. Like many of you, I was refreshing the page waiting to download Qwen 3.8 27B…

31
r/LocalLLaMA community 13d ago

[Paper] Intern-S2-Mobius: Foundation Model with Decoupled Knowledge and Reasoning

We introduce Mobius-v0, an architecture that comprises a globally shared Memory (FFN) that stores knowledge vectors and multiple Reasoners (Self-Attn) that iteratively achieve compositional reasoning. Using hidden states as cache and carrier, reasoners repeatedly query memory…

17
r/LocalLLaMA community 13d ago

tencent/EVIE-Preview-4.5B · Hugging Face

Overview EVIE-Preview-4.5B is a state-of-the-art multilingual Visual Document Retrieval (VDR) model built upon Qwen3.5-4B . It employs ColBERT-style late interaction with native 128-dimensional multi-vector token embeddings (4.54B parameters, BF16). By combining native…

37
r/LocalLLaMA community 13d ago

100$ worth of gpu runs qwen 3.8 27b at 7.39 t/s

Qwen 27b Q3_K_M 2x rx 580 8gb (~50$ each in my country, edge cases 60$ per gpu) gives us 16gb vram We used it on an old already existing ddr3 motherboard with 2 gpu slots(you can buy it ror around 200$ with 32 gb of ddr3 ram, a workstation xeon cpu and a workstation motherboard,…

20
r/LocalLLaMA community 13d ago

LLM's can't "jump" - a paper by Deepmind showing LLMs can't generate novel explanatory hypotheses

  submitted by   /u/juanviera23 [link]   [comments]

21
r/LocalLLaMA community 13d ago

Unpopular opinion : Qwen 3.8 27b is not an overthinker

Yes it uses a ton more reasoning tokens than 3.6 did But test in on the same tasks with the other chinese models, glm 5.3, deepseek v4 flash and pro, etc it's really similar, and they are needed The reality is, we're just frustrated because our hardware do not allow most of us…

19
r/LocalLLaMA community 13d ago

Petition to add a rule for people to add their DAMN quant levels to their posts

Every time I see a post about a newly released model, whether it be a comparison or shitting on it, I have to dig through the endless comments to see what quants they used and what their specs were. Its quite a common occurrence here in this sub to ask someone that's saying a…

25
r/LocalLLaMA community 13d ago

Ling 3.0 support merged into llama.cpp

Support for the new ling 3.0 models has been merged into llama.cpp: https://github.com/ggml-org/llama.cpp/pull/26608#event-29549472828 Ling tiny 8b1b - https://huggingface.co/inclusionAI/Ling-3.0-tiny Ling flash 124b5b - https://huggingface.co/inclusionAI/Ling-3.0-flash Both are…

6
r/LocalLLaMA community 13d ago

Qwen3.8-27B Q8_0 on Strix Halo is seriously impressive

Sorry for the slop, but I was impressed by this model as I have been testing Qwen3.8-27B Q8_0 locally on my ROG Flow Z13 (Ryzen AI Max+ 395, 128 GB unified memory) and this model was the only one who could made this short simulator (and I have tested a lot of models). Prompt:…

16
r/LocalLLaMA community 13d ago

Long Review: Qwen 3.8 27B is VERY good at tapping into it's real-world knowledge. It's "overthinking" brings it to Sonnet level performance with the potential for Opus level results.

Hi all! I finally just got around to testing out Qwen 3.8 27b. I'm using Unsloth's UD-Q8_K_XL quant as a sit-in replacement to Qwen 3.6 27b, same quant size. Wow -- this thing isn't messing around. I have many baseline test prompts to gauge the 'intelligence' and usability of…

28
r/LocalLLaMA community 13d ago

Stripe will reportedly acquire AI gateway startup OpenRouter for $7B+

another one ..   submitted by   /u/ab2377 [link]   [comments]

17
r/LocalLLaMA community 13d ago

How many tokens/second output are you getting with Qwen3.8-27B?

Trying to get a feel for where I stand. If you can list your relevant hardware and model used, that would be awesome. Here's mine: Model: Qwen3.8-27B-heretic-ara, Q5_K_M GGUF T/s : ~30-32 t/sec (I think, I'll verify in a bit) Hardware: 3090 GPU | 64 GBs DDR4 RAM | AMD 7950x CPU…

17
r/LocalLLaMA community 13d ago

…and I’m not afraid of losing my social credits.

  submitted by   /u/JLeonsarmiento [link]   [comments]

11
r/LocalLLaMA community 13d ago

[audio.cpp] Release 0.6: dots.tts, MiniMax-H3 text2audio (up to 3x realtime), MiniMax-Music3 (preview), and more new audio models. 5+ demos included.

Hi all :) audio.cpp release 0.6 has been out for a little while, so this is more of an update on what landed and what has been improving around it. 0.6 added 5 new model families: dots.tts, NeuTTS-2e, MuScriptor (Music to MIDI), MiniMax-H3, and SenseVoice-Small, bringing…

25
r/LocalLLaMA community 13d ago

Simon Willison: Qwen 3.8 27B is excellent, but it defaults to wildly overthinking things

I look to Simon for a broad survey of current LLM tech. Here's his review of playing with Qwen 3.8 27B . His comment on Mastodon was "I can't remember the last time I've had this much fun playing with a local model that runs on my own computers". BTW, the "wildly overthinking"…

30
r/LocalLLaMA community 13d ago

Qwen 3.8 27b vs 3.6 27b - how good is with a Turtle library.

Prompt: Provide complete working code for a realistic looking tree in Python using the Turtle graphics library and a recursive algorithm. Difference between 3.6 and 3.8 is huge!   submitted by   /u/Healthy-Nebula-3603 [link]   [comments]

6

OpenCode overrides the samplers for Qwen models to the wrong values

Linux Improves VRAM Management in 7.3 Kernel 🥳

Hugging Face just surpassed 3 million models on the Hub

DeepSeek V4 Flash 0731 on Strix Halo: draft model, n_max sweep, and a launch line that actually helps

Don't ignore llama.cpp RPC with old hardware. Results of a 5070 Ti and 1080 Ti over gigabit ethernet: it's actually functional.

Qwen 3.8 27b saved me $650+ in API costs this evening

Qwen 3.8 27B is faster than expected

Qwen 3.8 27b vs Deepseek Flash

tencent/UI-Mate-27B · Hugging Face

Qwen 3.8 27B xhigh vs medium small comparison (+ others for fun)

AA is the reason for Qwen3.8 27B shipped with xhigh

Qwen dev says not to wait for 35B-A3B

Optimizing Qwen3.6 / Qwen3.8-27B on 16GB VRAM: Complete Benchmark Results and Setup Guide (~30-50tps at 32k to 72k context)

CDW has bumped the MSRP of the RTX Pro 6000 from $16,000 to $19,999

Made this game in two prompts with Q4, Qwen 3.8 is amazing

Waiting for Qwen 3.8 35B A3B

Qwen 3.8 35b and 122b - We hope/wait/beg for models incessantly. But how do we actually give the lab more incentive to make it?

we benchmark models nobody actually runs

Qwen3.8-27B Uncensored Aggressive is out with K_P quants and HauhauCS FastMTP (up to 3.02x TG)!

Local agentic coding Benchmark : Qwen 3.8 27B (in many weights quants / cache quants / engine / reasoning effort) vs others.

Weirdly, no one talks about Temperature setting for the Qwen3.8 27b

I pushed Qwen3.8-27B to 99 tps single request and 1150 tps with a batch request on a RTX 3090

"Opus 4.8 thinks too much", "Muse Glimmer sits between Gemma and Qwen, that's boring", "Gemma 4 is too lazy"

Qwen 3.8 35bA3b wen?

llama.cpp adaptive MTP PR#27210

Qwen3.8 27B > Opus 5 Medium on Artificial Analysis Agentic Index

Qwen3.8 27B = GPT-5.6 Luna compressed into 27B

Artificial Analysis' Qwen3.8-27B benchmarks put it neck and neck with DeepSeek V4 and GPT-5.6 Luna Max

Why do people like coding harnesses like opencode etc instead of an IDE?

Ling 3.0 Tiny is the strongest, fastest and greatest model on my low end PC!

Qwen 3.8 27B Overthinking, It has to be done, it has to be overthinking to punch Opus 4.6

Deepseek Harnness - why is feels better

llama.cpp version v0.1.0 has been released

EXL3 seems to be fading from the r/LocalLLaMa consciousness, and while I suspected it, I'm surprised at this point in time.

After pushing 1M+ tokens through Qwen 3.8 27B, here is my optimal llama.cpp config for 16GB VRAM (73k Context, Agentic Coding)

[Paper] Intern-S2-Mobius: Foundation Model with Decoupled Knowledge and Reasoning

tencent/EVIE-Preview-4.5B · Hugging Face

100$ worth of gpu runs qwen 3.8 27b at 7.39 t/s

LLM's can't "jump" - a paper by Deepmind showing LLMs can't generate novel explanatory hypotheses

Unpopular opinion : Qwen 3.8 27b is not an overthinker

Petition to add a rule for people to add their DAMN quant levels to their posts

Ling 3.0 support merged into llama.cpp

Qwen3.8-27B Q8_0 on Strix Halo is seriously impressive

Long Review: Qwen 3.8 27B is VERY good at tapping into it's real-world knowledge. It's "overthinking" brings it to Sonnet level performance with the potential for Opus level results.

Stripe will reportedly acquire AI gateway startup OpenRouter for $7B+

How many tokens/second output are you getting with Qwen3.8-27B?

…and I’m not afraid of losing my social credits.

[audio.cpp] Release 0.6: dots.tts, MiniMax-H3 text2audio (up to 3x realtime), MiniMax-Music3 (preview), and more new audio models. 5+ demos included.

Simon Willison: Qwen 3.8 27B is excellent, but it defaults to wildly overthinking things

Qwen 3.8 27b vs 3.6 27b - how good is with a Turtle library.