r/LocalLLaMA

500 articles archived · Visit source ↗ · RSS

r/LocalLLaMA community 10d ago

Qwen 3.8 27B SlopCodeBench results

Howdy, I'm back again - running my favorite benchmark (it's still unsaturated for the time being so might as well!) previous runs a b https://github.com/michaelasper/benchmarks/blob/main/qwen3.8-27b-pi-on-slop-code-bench.md I ran this via OpenRouter because my mac would cry…

10
r/LocalLLaMA community 10d ago

Anyone NOT on full auto when coding with local LLMs?

Would love to know who's letting a 9B just go ham locally, haha But in all seriousness, how many of you are keeping to manual or manual-ish dev workflows?   submitted by   /u/BatPlack [link]   [comments]

23
r/LocalLLaMA community 10d ago

Fluid Simulation Qwen3.8 27B IQ3_XXS

After reading this tweet: https://x.com/scaling01/status/2089784644400976254 where the author says Qwen3.8 27B is not comparable to Opus, I had to try it out myself. First of: GPT 5.6 Sol (High) also failed the task and gave me a blank screen on the first try. And the author…

7
r/LocalLLaMA community 10d ago

I pushed Qwen3.8-27B limits again... Dflash2 - 134 tps on a RTX 3090

Edit: Title says 134 tps, it's actually 138 -- keep in mind my 3090 is power limited to 250w. Three days ago I released a hyper-optimized Qwen3.8-27B inference engine for an RTX 3090 (82 tps single request, 672 peak), and yesterday's update took it to ~114 tps single-user /…

30
r/LocalLLaMA community 10d ago

We quantized the new Ornith 1.5 9B and 35B-A3B

ornith lab dropped new ornith 1.5 today, a 9B dense with vision and a 35B-A3B MoE, both MIT, trained on a loop that generates its own tasks. in addition there was giant 397b model, but we didn't quantize it (but if you want to try - we will do it) we made our AD (Atomic Dynamic)…

21
r/LocalLLaMA community 10d ago

Ornith-1.5 9B might not be bad at all

(my own experience) I have a mid-range gaming laptop from 2024 with 4GB VRAM and 16GB RAM. There is nothing that I can run other than small models at humble t/s. I mostly use the models to generate scripts for medical physics research (don't worry, I verify them before any use,…

9
r/LocalLLaMA community 11d ago

Reverse-Engineering the RK3588 NPU: Building an Open Compiler to Run GPT-2 at 36 tok/s

Last year I posted about hacking the RK3588 NPU to run one vision encoder ( previous post ). This year I opened the whole thing up: reverse-engineered the register format, built an open compiler + runtime, and now GPT-2 and SigLIP run from PyTorch, ONNX, and JAX, no vendor SDK.…

30
r/LocalLLaMA community 11d ago

DFlash2 speeds Qwen 3.8 27B up to 4 times

llama.cpp pr #27342 adds dflash2, so i rented an rtx 6000 and ran the same four prompts through four decoding setups on qwen3.8 27B median results over the four tasks: baseline 47.4 tok/s mtp 114.7 tok/s dflash 99.3 tok/s dflash2 140.6. tok/s so on average 3x for dflash2 though…

33
r/LocalLLaMA community 11d ago

I might have found the perfect config parameters for qwen 3.8 27b

Hello everyone, tried so hard to optimize my config and finally I simply get up to 70 t/s with q6 variant. And wanted to share with you guys so that other people with the same setup can enjoy. Please check out and see if that improves your performance in any kind of way. Also…

15
r/LocalLLaMA community 11d ago

LFM 2.5 QAD

https://x.com/liquidai/status/2090078070929760295 https://huggingface.co/LiquidAI/LFM2.5-2.6B-GGUF   submitted by   /u/jacek2023 [link]   [comments]

5
r/LocalLLaMA community 11d ago

Introducing Qwen3.8-27B Dynamic v3 Unsloth GGUFs

Hey everyone! We’re releasing new Qwen3.8-27B GGUFs with 10% higher accuracy for the same size. This uses a new version of Dynamic v3.0 Unsloth Dynamic V3 outperforms others by >10% on Div-300, KLD & more benchmarks. We also release 1-bit quants that retain 77% accuracy. Run on…

34
r/LocalLLaMA community 11d ago

AntLing’ve open-sourced 6 Base Model checkpoints for Ling-3.0-tiny & Ling-3.0-flash, covering pre-trained, mid-trained, and WSM-merged stages.

None has undergone post-training, giving researchers flexible starting points for continued pre-training, fine-tuning, and further research. Two key highlights: - They use WSM to replace LR decay with weighted checkpoint merging, making the training process better suited for…

18
r/LocalLLaMA community 11d ago

ling 3.0 flash/tiny base models

https://huggingface.co/inclusionAI/Ling-3.0-tiny-base https://huggingface.co/inclusionAI/Ling-3.0-flash-base-midtrain https://huggingface.co/inclusionAI/Ling-3.0-flash-base-30T https://huggingface.co/inclusionAI/Ling-3.0-tiny-base-midtrain…

25
r/LocalLLaMA community 11d ago

NVFP4 on VOLTA! Despite being built for Blackwell, I made four 2017 V100s run Qwen 3.8 NVFP4 natively and match my $6000 RTX 5090.

Four Tesla V100s from 2017 matched my RTX 5090 on single-request Qwen 3.8 decode. Repo: https://github.com/dnv2003/v100-skinny https://i.redd.it/5ws2ak3uqckh1.gif The 5090 was not being held back. It ran NInfer , a specialist engine built to make this exact model as fast as…

29
r/LocalLLaMA community 11d ago

Ornith-1.5 (397B [DeepSWE 56], 35B-A3B, 9B)

Aloha! 🌺Introducing Ornith-1.5, a family of open-source LLMs spanning 9B Dense, 35B MoE, and 397B MoE, trained with self-improving strategies. It achieves state-of-the-art performance among open-source models of comparable size and delivers performance comparable to Claude Opus…

28
r/LocalLLaMA community 11d ago

Ornith 1.5: 9B dense and 35B/397B MoEs

  submitted by   /u/tarruda [link]   [comments]

13
r/LocalLLaMA community 11d ago

We have Q3.8 35B at home: 3x new Ornith 1.5 released

Anyone tried them yet? https://huggingface.co/ornith-ai/Ornith-1.5-9B https://huggingface.co/ornith-ai/Ornith-1.5-35B-A3B https://huggingface.co/ornith-ai/Ornith-1.5-397B https://huggingface.co/ornith-ai/Ornith-1.5-9B-GGUF https://huggingface.co/ornith-ai/Ornith-1.5-35B-A3B-GGUF…

9
r/LocalLLaMA community 11d ago

updated unsloth/Qwen3.8-27B-GGUF · Hugging Face

looks like GGUF files were just updated   submitted by   /u/jacek2023 [link]   [comments]

34
r/LocalLLaMA community 11d ago

Finally found a really solid suno-like minimax music UI!!

Been messing with minimax music gen lately. I really like Suno and was basically looking for something that gave me a similar workflow for minimax. I got completely sick of running everything through the CLI. I went digging on github, sorted by recent, and took a gamble on this…

8
r/LocalLLaMA community 11d ago

Waiting for a 122B because of world knowledge?

Any LLM will hallucinate the world knowledge, even a 3T model. Use a 4B with a kiwix skill and local Wikipedia, 50gb and no more hallucinated world knowledge. Ask your coding agent to build your own, with your rules and eventual fallback access to internet knowledge for what's…

29
r/LocalLLaMA community 11d ago

[Open PR] llama : add --n-cpu-ffn option by John-194 · Pull Request #26622 · ggml-org/llama.cpp

PR by u/Stainless-Bacon 👍 It would be handy & awesome to have options --n-cpu-ffn / --cpu-ffn for Dense models like how we have --n-cpu-moe / --cpu-moe for MOE models . Also check his threads: On PR : llama.cpp CPU offload optimizations On Running Qwen3.8-27B with above…

18
r/LocalLLaMA community 11d ago

Stop Anthropomorphisizing Intermediate Tokens: Qwen3.8 doesn't "overthink"

Intermediate tokens, called "thinking" or "reasoning" actually are nothing like it. Humans do step-by-step reasoning leading to the conclusion. LLMs use intermediate traces to augment their prompt . This explains why sometimes the answer is very good but the "reasoning" is…

21
r/LocalLLaMA community 11d ago

Am I doing something wrong? Qwen 3.8 27B seems useless for agentic coding

I have been using local models on/off for like 2 years or so but never really used them extensively because the closed ones were always much better. Once Qwen 3.8 27B was released I decided to give it another serious try. I configured Cline and ZooCode as VSCode addons,…

29
r/LocalLLaMA community 11d ago

[2511.07885] Intelligence per Watt: Measuring Intelligence Efficiency of Local AI

  submitted by   /u/pscoutou [link]   [comments]

38
r/LocalLLaMA community 11d ago

Thoughts About Scaling Law - Z.ai

Thoughts About Scaling Law Scaling, but not only of parameters. Every model release now ends with the same question: how many parameters? It isn't a question that can be answered on its own. Parameter count is only meaningful alongside three others — how much data you have,…

14
r/LocalLLaMA community 11d ago

Qwen3.8-27B on 2x 3090 + vLLM + DFlash2: 218 tok/s single request

I hacked this together so there's probably more on the table in terms of performance. Measured with the Club-3090 canonical bench suite (bench.sh, 3 warmups + 5 measured runs, temp 0.6 / top_p 0.95 / top_k 20). Prefill: 1342 tok/s @ 10k, 628 tok/s @ 90k Spec-decode: 7 draft…

6
r/LocalLLaMA community 11d ago

New midsize Qwen 3.8 model coming next week (hopefully) according to community manager!

Community manager mentioned this in the Qwen Ambassador Discord, put an X reaction on someone asking for 35B... and said We'll have a new midsize open weight model coming next week (hopfully), This midsize model won't provide early access due to the schedule Thinking it's going…

12
r/LocalLLaMA community 11d ago

Ling-3.0-tiny is a very interesting model. Run on NVIDIA Orin Nano Super 8GB at 128K context with IQ4_NL quant.

I have been searching for suitable model to run on my 8GB RAM toy, NVIDIA Orin Nano Super 8GB. This little toy was priced at $249 earlier this year (not any more), and pulls very little power when idle. It was an interesting device that suitable for an agent to host on. It is…

31
r/LocalLLaMA community 11d ago

What is the best Qwen3.8 27b Abliterated version out there?

I'm trying to get a model to reverse engineer / decompile or otherwise reverse to source some of my old c,c++, pascal, and asm demo programs I made from decades ago and I'm constantly met with refusals. It is highly annoying. Does anybody know of a good abliterated/uncensored…

24
r/LocalLLaMA community 11d ago

I tested DFlash2 for Qwen3.8 27B on a 5090

Here's the DFlash2 announcement , and I was pretty excited for this after trying out DSpark on llama.cpp a few days ago and being somewhat disappointed that it wasn't really working. Anyways, I spent a while setting it up (you need to rebuild llama.cpp with pr #27342). Here's my…

19
r/LocalLLaMA community 11d ago

GLM5.3 Artificial Analysis Benchmarks

  submitted by   /u/anderspitman [link]   [comments]

33
r/LocalLLaMA community 11d ago

DFlash 2: Keep Drafting Parallel

  submitted by   /u/coder543 [link]   [comments]

23
r/LocalLLaMA community 11d ago

DFlash 2 available for Qwen 3.8 27B and Muse Glimmer

Apparently a second version of DFlash from the original authors of DFlash GGUF quants are already made available with an accompanying llama.cpp PR: https://github.com/ggml-org/llama.cpp/pull/27342   submitted by   /u/rerri [link]   [comments]

10
r/LocalLLaMA community 11d ago

Alibaba's RISC-V CPU, XuanTie C950, Runs Qwen-3.8 27B at 30 tps

Who needs GPUs?   submitted by   /u/DeltaSqueezer [link]   [comments]

25
r/LocalLLaMA community 11d ago

local models fear my tests

am I the only one who does this lol   submitted by   /u/close_Meal6005 [link]   [comments]

20
r/LocalLLaMA community 12d ago

1-bit / 2-bit / Ternary / Bitnet Models - Updates & Tracking

Bonsai / Ternary Bonsai During April Bonsai came with bunch of models .... 1-bit & 1.58-bit(Ternary) versions. And last month(July) they released 27B models in same versions. Last month itself, Bonsai-27B was able to run on all backends mainline. But Ternary-Bonsai-27B was not…

38
r/LocalLLaMA community 12d ago

and here we are

  submitted by   /u/johnnyApplePRNG [link]   [comments]

6
r/LocalLLaMA community 12d ago

Ling-3.0 (BailingMoE3) lands in llama.cpp mainline - Quick benchmarks on Intel Arc B580

Finally llama.cpp now officially supports Ling-3.0! (Starting from build b10472 +) If you want to run them locally, bartowski has already released the GGUF imatrix quantizations for both models: - Ling-3.0-tiny (8B) - Ling-3.0-flash (127B) After quite a while, PR #26608 has…

33
r/LocalLLaMA community 12d ago

Idea: massively compress Qwen 3.8 KV cache by using a single bit for the token "wait"

Not even sure if I'm joking, my thinking history is about 50% "wait".   submitted by   /u/pixelpoet_nz [link]   [comments]

14
r/LocalLLaMA community 12d ago

Memory prices climb 500% in 12 months, up to 10x the lowest ever tracked prices - 128GB of DDR5 now $3,399

  submitted by   /u/johnnyApplePRNG [link]   [comments]

37
r/LocalLLaMA community 12d ago

Qwen3.8 2.4T open weights made a Call of Duty clone

Qwen released the 2.4T Max weights and I was curious how well it can re-create COD in one prompt I ran the model on a rented B200 cluster and used roughly 1.1M output tokens over a 5 hour time span Realistically barely anyone can run this model locally, but the fact that it's…

18
r/LocalLLaMA community 12d ago

I pushed Qwen3.8-27B to 124 tps on a single request on a RTX 3090

Two days ago I released a hyper-optimized Qwen3.8-27B inference engine for an RTX 3090 (82 tps single request, 672 peak) - yesterday's update took that to 99 tps single-user / ~1,000 tps at 64 concurrent. Since then I've focused on the single-request number, again without…

31
r/LocalLLaMA community 12d ago

Qwen3.8-27B: slower tokens, faster and better results

  submitted by   /u/surreal_tournament [link]   [comments]

30
r/LocalLLaMA community 12d ago

Running DeepSeek V4 Flash Q4_K_XL at ~100 tok/s prompt processing on 4× RTX 3060 12GB

I managed to run the 143–144 GiB DeepSeek-V4-Flash-0731 UD-Q4_K_XL GGUF on four RTX 3060 12GB cards while keeping a 360k–376k context window. Hardware: CPU: Intel Core i9-10920X, 12C/24T RAM: 128 GB DDR4-3200, quad-channel GPU: 4× NVIDIA RTX 3060 12GB Total VRAM: 48 GB Storage:…

25
r/LocalLLaMA community 12d ago

If the weights never change, is it really recursive self-improvement?

https://preview.redd.it/e0ydm43a55kh1.png?width=2902&format=png&auto=webp&s=8c9b88ff5e4157811e8996ba5a1e96cc55c8ae6a This paper is using a much narrower definition of recursive self-improvement than the phrase usually suggests. AQuA stores validated evidence in a persistent…

16
r/LocalLLaMA community 12d ago

Is Ling 3 tiny underrated for its size?

I was checking out benchmarks of this model and apparantly better than Qwen3.5 9b reasoning across the bench on artificial analysis. I have used the 9b model for variety of stuff and it has been amazing, but if this is better then why not switch. I am downloading it rn to test…

6
r/LocalLLaMA community 12d ago

What sandbox are you all using for AI agents?

Hi everyone, I use AI agents to code and manage my personal files heavily. However, I am worried that the agents may accidentally delete some important files on my machine, outside of the defined scope. I am wondering what sandboxes people are using. I am looking for a solution…

27
r/LocalLLaMA community 12d ago

OpenCode overrides the samplers for Qwen models to the wrong values

This is invisible with llama.cpp or derivatives, but ninfer helpfully logs the sampler settings on each request and auto-configures the correct ones for the model. Basically, OpenCode will always send top-p=1.0 (which means there is no filtering of low-probability tokens except…

17
r/LocalLLaMA community 12d ago

Linux Improves VRAM Management in 7.3 Kernel 🥳

  submitted by   /u/johnnyApplePRNG [link]   [comments]

17
r/LocalLLaMA community 12d ago

Hugging Face just surpassed 3 million models on the Hub

From Hugging Face on 𝕏: https://x.com/huggingface/status/2089673018737869242?s=20   submitted by   /u/Nunki08 [link]   [comments]

30

Qwen 3.8 27B SlopCodeBench results

Anyone NOT on full auto when coding with local LLMs?

Fluid Simulation Qwen3.8 27B IQ3_XXS

I pushed Qwen3.8-27B limits again... Dflash2 - 134 tps on a RTX 3090

We quantized the new Ornith 1.5 9B and 35B-A3B

Ornith-1.5 9B might not be bad at all

Reverse-Engineering the RK3588 NPU: Building an Open Compiler to Run GPT-2 at 36 tok/s

DFlash2 speeds Qwen 3.8 27B up to 4 times

I might have found the perfect config parameters for qwen 3.8 27b

LFM 2.5 QAD

Introducing Qwen3.8-27B Dynamic v3 Unsloth GGUFs

AntLing’ve open-sourced 6 Base Model checkpoints for Ling-3.0-tiny & Ling-3.0-flash, covering pre-trained, mid-trained, and WSM-merged stages.

ling 3.0 flash/tiny base models

NVFP4 on VOLTA! Despite being built for Blackwell, I made four 2017 V100s run Qwen 3.8 NVFP4 natively and match my $6000 RTX 5090.

Ornith-1.5 (397B [DeepSWE 56], 35B-A3B, 9B)

Ornith 1.5: 9B dense and 35B/397B MoEs

We have Q3.8 35B at home: 3x new Ornith 1.5 released

updated unsloth/Qwen3.8-27B-GGUF · Hugging Face

Finally found a really solid suno-like minimax music UI!!

Waiting for a 122B because of world knowledge?

[Open PR] llama : add --n-cpu-ffn option by John-194 · Pull Request #26622 · ggml-org/llama.cpp

Stop Anthropomorphisizing Intermediate Tokens: Qwen3.8 doesn't "overthink"

Am I doing something wrong? Qwen 3.8 27B seems useless for agentic coding

[2511.07885] Intelligence per Watt: Measuring Intelligence Efficiency of Local AI

Thoughts About Scaling Law - Z.ai

Qwen3.8-27B on 2x 3090 + vLLM + DFlash2: 218 tok/s single request

New midsize Qwen 3.8 model coming next week (hopefully) according to community manager!

Ling-3.0-tiny is a very interesting model. Run on NVIDIA Orin Nano Super 8GB at 128K context with IQ4_NL quant.

What is the best Qwen3.8 27b Abliterated version out there?

I tested DFlash2 for Qwen3.8 27B on a 5090

GLM5.3 Artificial Analysis Benchmarks

DFlash 2: Keep Drafting Parallel

DFlash 2 available for Qwen 3.8 27B and Muse Glimmer

Alibaba's RISC-V CPU, XuanTie C950, Runs Qwen-3.8 27B at 30 tps

local models fear my tests

1-bit / 2-bit / Ternary / Bitnet Models - Updates & Tracking

and here we are

Ling-3.0 (BailingMoE3) lands in llama.cpp mainline - Quick benchmarks on Intel Arc B580

Idea: massively compress Qwen 3.8 KV cache by using a single bit for the token "wait"

Memory prices climb 500% in 12 months, up to 10x the lowest ever tracked prices - 128GB of DDR5 now $3,399

Qwen3.8 2.4T open weights made a Call of Duty clone

I pushed Qwen3.8-27B to 124 tps on a single request on a RTX 3090

Qwen3.8-27B: slower tokens, faster and better results

Running DeepSeek V4 Flash Q4_K_XL at ~100 tok/s prompt processing on 4× RTX 3060 12GB

If the weights never change, is it really recursive self-improvement?

Is Ling 3 tiny underrated for its size?

What sandbox are you all using for AI agents?

OpenCode overrides the samplers for Qwen models to the wrong values

Linux Improves VRAM Management in 7.3 Kernel 🥳

Hugging Face just surpassed 3 million models on the Hub