r/LocalLLaMA

500 articles archived · Visit source ↗ · RSS

r/LocalLLaMA community 13d ago

Stripe will reportedly acquire AI gateway startup OpenRouter for $7B+

another one ..   submitted by   /u/ab2377 [link]   [comments]

17
r/LocalLLaMA community 13d ago

How many tokens/second output are you getting with Qwen3.8-27B?

Trying to get a feel for where I stand. If you can list your relevant hardware and model used, that would be awesome. Here's mine: Model: Qwen3.8-27B-heretic-ara, Q5_K_M GGUF T/s : ~30-32 t/sec (I think, I'll verify in a bit) Hardware: 3090 GPU | 64 GBs DDR4 RAM | AMD 7950x CPU…

17
r/LocalLLaMA community 13d ago

…and I’m not afraid of losing my social credits.

  submitted by   /u/JLeonsarmiento [link]   [comments]

11
r/LocalLLaMA community 13d ago

[audio.cpp] Release 0.6: dots.tts, MiniMax-H3 text2audio (up to 3x realtime), MiniMax-Music3 (preview), and more new audio models. 5+ demos included.

Hi all :) audio.cpp release 0.6 has been out for a little while, so this is more of an update on what landed and what has been improving around it. 0.6 added 5 new model families: dots.tts, NeuTTS-2e, MuScriptor (Music to MIDI), MiniMax-H3, and SenseVoice-Small, bringing…

25
r/LocalLLaMA community 13d ago

Simon Willison: Qwen 3.8 27B is excellent, but it defaults to wildly overthinking things

I look to Simon for a broad survey of current LLM tech. Here's his review of playing with Qwen 3.8 27B . His comment on Mastodon was "I can't remember the last time I've had this much fun playing with a local model that runs on my own computers". BTW, the "wildly overthinking"…

30
r/LocalLLaMA community 13d ago

Qwen 3.8 27b vs 3.6 27b - how good is with a Turtle library.

Prompt: Provide complete working code for a realistic looking tree in Python using the Turtle graphics library and a recursive algorithm. Difference between 3.6 and 3.8 is huge!   submitted by   /u/Healthy-Nebula-3603 [link]   [comments]

6
r/LocalLLaMA community 13d ago

Dario Amodei defends his policy proposals, warns open weights won't decentralize power, endorses pre-launch vetting, says real accomplishments will earn trust

  submitted by   /u/f0urxio [link]   [comments]

17
r/LocalLLaMA community 13d ago

Anyone else get a kick out of Qwen 3.8 27B Reasoning Dialogue?

I've been paying attention to the reasoning because I'm still evaluating the model, and I've just noticed that sometimes I get a kick out of the way this model's internal monologue seems to play out sometimes. Like I've seen it get genuinely frustrated with itself and express…

27
r/LocalLLaMA community 13d ago

Qwen 3.8 9b?

  submitted by   /u/Thatisverytrue54321 [link]   [comments]

25
r/LocalLLaMA community 14d ago

Qwen3.8-27b on RTX 3090 - 82 tps single request, up to 672 tps peak

Hi, After a long night of optimizations, I believe I have made the fastest inference engine for Qwen3.6-28B on a 3090. Quick metrics: - 250w power capped - Up to 195k context (ships with 150k for safety though) - 82 tps single request, 417 tps sustained with 64 concurrent -…

23
r/LocalLLaMA community 14d ago

Why are RTX 6000 PROs still getting bought at 16000+ USD? And who are buying them?

Hello guys, hoping you're doing well. I bring this discussion since I have noticed on internet, be USA or EU, RTX 6000 PROs at 16000USD or more are still getting bought. Even here on Chile, the other day they were in stock at 20000-21000USD post 19% tax and they lasted a few…

8
r/LocalLLaMA community 14d ago

Qwen3.8 27B Q2 vs Q3 vs Qwen3.6 35B-A3B MoE on 12GB VRAM

Did a quick local test because I wanted to see what is actually usable on my 12GB laptop GPU. I tested the newer Qwen3.8 27B dense files at Q2 and Q3, then compared them against Qwen3.6 35B-A3B MoE. Hardware: RTX 5070 Ti Laptop, 12GB VRAM Backend: llama.cpp CUDA Settings: 4k…

12
r/LocalLLaMA community 14d ago

Qwen 3.8 2.4T at 288k tokens/s on Nvidia GB300 NVL72

https://developer.nvidia.com/blog/serve-qwen3-8-2-4t-a95b-a-2-4t-parameter-model-with-configurable-reasoning-on-nvidia-gb300-nvl72/ 4k tokens per second per GPU of which there are 72. 350 tokens per second per user "Without additional model tuning, the model achieves a…

13
r/LocalLLaMA community 14d ago

Qwen 3.8 distillations

https://x.com/i/status/2088993948983246906 Not tested by me in any way :)   submitted by   /u/jacek2023 [link]   [comments]

13
r/LocalLLaMA community 14d ago

Based on an accelerating frontier -> local trajectory, expect a ~30b param 'Mythos at home' by as soon as Jan 2027 (rationalisation below)

Including the rationalisation for the data below - this is a more robust version of an earlier post I did similar to this - explaining below: How I chose the comparisons The basic question I’m trying to answer is: when did an open model small enough to run on high-end consumer…

29
r/LocalLLaMA community 14d ago

Let’s all thank Georgi Gerganov who gave use llama.cpp

I was looking into the story a bit further earlier. Very interesting. Couldn’t have done it without him   submitted by   /u/on_line187 [link]   [comments]

10
r/LocalLLaMA community 14d ago

Koboldcpp v1.119 released

  submitted by   /u/Fcking_Chuck [link]   [comments]

7
r/LocalLLaMA community 14d ago

Qwen3.8-27B Hybrid IQ4_XS quantization for 16GB gang

  submitted by   /u/Johnny_Rell [link]   [comments]

11
r/LocalLLaMA community 14d ago

Newer commits removed the Qwen 35B

In this commits, the 35B model was removed. Looks like it's confirming the 35B model won't get released. I think they need to be made aware how big the 35 moe is widely used. Think need to make noise on theyre X, huggingface and online places. If they dont know there's no need…

24
r/LocalLLaMA community 14d ago

Let's play : guess the model

Single prompt "make me a 3d scene of an anime girl" (stolen from someone on discord, I like it a lot since it seems like every model struggles a lot with it) Everything is one shot. No vision, no iterations, no corrections, no going in the browser. Essentially depends on…

33
r/LocalLLaMA community 14d ago

Quick PSA: Qwen3.8-27B reasoning effort vs reasoning budget in llama.cpp

If you are using llama-server with their web-ui for testing, keep in mind, that the reasoning selector is just a reasoning budget aka a hard cap and has, at least to my knowledge, nothing at all to do with Qwen3.8-27B's native reasoning effort capability! Selecting any value for…

34
r/LocalLLaMA community 14d ago

The dream is to reach 200GB VRAM

Step 1) Find 16k ASAP before it goes up to 20k after a few months Step 2) Buy RTX PRO 6000 (MAXQ) Step 3) Remove RTX PRO 5000 in pcie_1 slot. Replace w/ RTX PRO 6000 Step 4) Buy a NVME to PCIE converter and HPPLEX 500W then move RTX PRO 5000 there Step 5) Power limit RTX PRO…

5
r/LocalLLaMA community 14d ago

Huihui-ai Qwen 3.8 Ablit Available

At hugging face, this is the model I use for most of my analysis that most of my services have to not be refused. previous versions are quite good. It seems this might have dropped today and pulling right now!   submitted by   /u/Frizzy-MacDrizzle [link]   [comments]

38
r/LocalLLaMA community 14d ago

Qwen3.8-27B-int4-AutoRound (18GB) - with working MTP spec decode

  submitted by   /u/BusinessMud9586 [link]   [comments]

25
r/LocalLLaMA community 14d ago

Qwen 3.8 27b with DSH(DeepSeek Harness) is Amazing!! Experiences so far and perfomance.

https://preview.redd.it/wkg27e152qjh1.png?width=853&format=png&auto=webp&s=2e3f8b11ea6393041f501e95c5835f9bea0245dd So ive been trying different harnesses and coding agents with the new qwen 3.8 , and after trying out many ive been mostly impressed by deekseek harness , paired…

33
r/LocalLLaMA community 14d ago

Qwen 3.8 27b - Acquarium Panel Failure - Q8_K_XL Unsloth

After: https://www.reddit.com/r/LocalLLaMA/comments/1vckcue/ds4_flash_0731_acquarium_panel_failure_q3_k_xl/ start C:\llm\llamaM6\build\bin\llama-server.exe --model "F:\models\Qwen3.8-27B-UD-Q8_K_XL.gguf" --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --presence-penalty 0.0…

10
r/LocalLLaMA community 14d ago

Genie-style playable world model running 720p at 16 FPS on a single 5090 in 19GB VRAM

  submitted by   /u/juanviera23 [link]   [comments]

35
r/LocalLLaMA community 14d ago

Paper claims RL for reasoning only changes 1-3% of tokens, and they replicate the gains without RL at ~1000x less compute

  submitted by   /u/juanviera23 [link]   [comments]

5
r/LocalLLaMA community 14d ago

Qwen3.8 27B reasoning effort low/medium/xhigh comparison

I did a short test of the different reasoning efforts, since on default xhigh the model thinks a lot . Not very scientific, just a quick "generate an SVG of a pelican on a bicycle" prompt with 3 different seeds. I think the result is interesting none the less: xhigh gives *much\…

28
r/LocalLLaMA community 14d ago

I'm still running Qwen 3.5 122B. Should I switch to Qwen 3.8 27B?

I see everyone gushing over 3.8, I get the impression people find it drastically better than previous Qwen models, but I can't believe it could be better than 3.5 122B. Is it?   submitted by   /u/MackThax [link]   [comments]

28
r/LocalLLaMA community 14d ago

Is ternary (1.58-bit) LLMs making a come back?

I'm just thinking, ever since microsoft announced bitnet, this sub (and myself) has been hoping for massive ternary models. In the last month alone, prismML dropped 27B ternary (though I've read community experience suggested it sometimes didn't hold up to it's benchmarks),…

33
r/LocalLLaMA community 14d ago

If you are at the lowest budget, which you can think of.Which hardware would you recommend to run? qwen 3.8 27b oWith like 50 tokens per second. I currently have a RTX 5070 Ti.

  submitted by   /u/InternationalGap3698 [link]   [comments]

17
r/LocalLLaMA community 14d ago

Anyone still use turboquant?

I haven’t used it in a while, is it still worth it?   submitted by   /u/Adventurous-Gold6413 [link]   [comments]

36
r/LocalLLaMA community 14d ago

Qwen3.8-27B abliterated FP8: refusal 64–99% → 0–6%, and MMLU/GSM8K move less than 1.3 points

Been reading the eval table on the abliterated Qwen3.8-27B FP8 build instead of the release notes. It's published as red-team material, disclaimer and all, so the numbers are the interesting part. Refusal across the usual harmful-instruction sets (AdvBench, HarmBench,…

12
r/LocalLLaMA community 14d ago

Anyone managed to get Qwen 3.8 27B running smoothly on vLLM? Can't get rid of endless thinking

Title pretty much says it all. I’ve deployed Qwen 3.8 27B using vLLM on an RTX 6000 Pro (tried multiple vLLM releases and launch recipes), but I can't get it into a usable state because of crazy long reasoning passes. Regardless of the thinking effort setting (xhigh, medium, or…

22
r/LocalLLaMA community 14d ago

5090: Windows or Linux for Qwen3.8.27b

I've got a dedicated AI rig sitting here with a RTX 5090 and 96GB RAM and for the past few years have been using Windows 11 and primarily LM Studio, but have also used vLLM, llama.cpp and Ollama. With Qwen3.8.27b I want to get the most out of this model. I get the feeling from…

14
r/LocalLLaMA community 14d ago

How many people have 24gb over gpu here?

I was surprised by the fact that the qwen 3.8 27b download count is about 1 million (globally). This means that even on this subreddit, very few people have used 27b. At most 50k–100k active users, and once you break down the hardware distribution, 8GB, 16GB, 24GB, 32GB cards,…

5
r/LocalLLaMA community 14d ago

Show-off Saturday: Intel Arc B140 build.

My local inference build, with: ASUS WS C422 PRO/SE 10-core Xeon W-2255 64GB ECC RAM 64GB VRAM Pimped case with TurboLEDz indicating the frequencies of the 10 xeon cores. Running llama.cpp with SYCL back-end. Khronos-stack and MESA stack all built from git sources, running on…

37
r/LocalLLaMA community 14d ago

Qwen3.8-27B vs Qwen3.6-27B writing ray-tracers in BASIC

one of my llm hobbies is re-creating graphics demos i used to write in BASIC in the late 1980s. i slopped together an agentic harness and a basic-to-js transpiler in a web page i've been playing with for a few months. the agent can write basic programs, run them, examine the…

21
r/LocalLLaMA community 14d ago

Need assistance in getting my 4 AMD GPU setup running on a Z10PA-U8 mainboard

I'm trying to run 4x AMD RX 6900 XT and an NVIDIA GT 710 (x8) display card on an ASUS Z10PA-U8 mainboard and Ubuntu Server 24.04. But so far this doesn't work so well. I have an old GA-Z77-DS3H mainboard which has 1 PCIe x16 (Gen3), 2 PCIe x1 (Gen2) and 1 PCIe x16 (Gen2) slots.…

29
r/LocalLLaMA community 14d ago

SOTA Apple Silicon Inference (August 15, 2026)

This is a HANDWRITTEN post. I spent way too much time trying to get fast inference on Apple Silicon. This post is for people who want to know what's the latest on running local models on their mac, and why they may not be seeing the performance others in the community claim.…

34
r/LocalLLaMA community 14d ago

llama.cpp Windows Manager

https://github.com/alekk89/llama-cpp-windows-manager It has been three months since I first shared my personal solution for running llama.cpp on Windows, and the project has evolved considerably since then. The goal was to create a simple visual interface for managing llama.cpp…

19
r/LocalLLaMA community 14d ago

The perfect way for Google to screw over OAI and Anthropic is by releasing a 120B dense multimodal Gemma model

The two leading labs are already feeling extremely threatened by Qwen & friends, however I think there are tons of enterprises and organizations in the West that don't feel comfortable using Chinese models. I believe these orgs would be all over a near-frontier open-weight model…

35
r/LocalLLaMA community 14d ago

club-5060ti refresh: tested RTX 5060 Ti presets, a proper high-context harness, and Qwen3.8 27B

Quick update on the RTX 5060 Ti local LLM repo. It has changed quite a bit since my previous posts. The project started as a collection of practical notes and benchmark results. That was useful, but as the dataset grew it became harder to answer the question most people actually…

10
r/LocalLLaMA community 15d ago

Which Harness for Local Coding (Qwen 3.8 27b) do you Recommend?

I know there are many posts about this, but I would explicitly like to know the harness names instead of having a general discussion. Let's see what the community actually uses! For others, please comment below. Feel free to say why you prefer yours. Thanks View Poll  …

36
r/LocalLLaMA community 15d ago

Agentic harness for small models

Hello; I'm a semi-beginner at local AI. I've been experimenting with this tech for a while, and I still haven't found a proper harness that fits my models, hardware, and needs. My use case is pretty simple: web search, fetching, and browser use. Summarizing websites and having…

29
r/LocalLLaMA community 15d ago

RTX Pro 4500 vs. A100 (x4) - purchase being made today

Need some help from the community on this server build: RTX Pro 4500 x4 ($3,299) or used 40GB A100 x4 off eBay (around $4,000/ea). The RTX Pro 4500 means I have to go the route of getting a PCIe switch, inviting a fair amount of complexity. A100 means I just need to find NVLink…

18
r/LocalLLaMA community 15d ago

The duality of man

Thoughts on the new model?   submitted by   /u/Tasty-Ad-3753 [link]   [comments]

31
r/LocalLLaMA community 15d ago

US to tell partners they must pick sides in AI race with China

  submitted by   /u/johnnyApplePRNG [link]   [comments]

13
r/LocalLLaMA community 15d ago

Best llm at understaning middle eastern languages? Arabic/Persian etc

From my testings, gemma 4 26ba4 and qwen 27b are doing great But curious if someone knows a better option   submitted by   /u/Whole_Alternative_18 [link]   [comments]

15

Stripe will reportedly acquire AI gateway startup OpenRouter for $7B+

How many tokens/second output are you getting with Qwen3.8-27B?

…and I’m not afraid of losing my social credits.

[audio.cpp] Release 0.6: dots.tts, MiniMax-H3 text2audio (up to 3x realtime), MiniMax-Music3 (preview), and more new audio models. 5+ demos included.

Simon Willison: Qwen 3.8 27B is excellent, but it defaults to wildly overthinking things

Qwen 3.8 27b vs 3.6 27b - how good is with a Turtle library.

Dario Amodei defends his policy proposals, warns open weights won't decentralize power, endorses pre-launch vetting, says real accomplishments will earn trust

Anyone else get a kick out of Qwen 3.8 27B Reasoning Dialogue?

Qwen 3.8 9b?

Qwen3.8-27b on RTX 3090 - 82 tps single request, up to 672 tps peak

Why are RTX 6000 PROs still getting bought at 16000+ USD? And who are buying them?

Qwen3.8 27B Q2 vs Q3 vs Qwen3.6 35B-A3B MoE on 12GB VRAM

Qwen 3.8 2.4T at 288k tokens/s on Nvidia GB300 NVL72

Qwen 3.8 distillations

Based on an accelerating frontier -> local trajectory, expect a ~30b param 'Mythos at home' by as soon as Jan 2027 (rationalisation below)

Let’s all thank Georgi Gerganov who gave use llama.cpp

Koboldcpp v1.119 released

Qwen3.8-27B Hybrid IQ4_XS quantization for 16GB gang

Newer commits removed the Qwen 35B

Let's play : guess the model

Quick PSA: Qwen3.8-27B reasoning effort vs reasoning budget in llama.cpp

The dream is to reach 200GB VRAM

Huihui-ai Qwen 3.8 Ablit Available

Qwen3.8-27B-int4-AutoRound (18GB) - with working MTP spec decode

Qwen 3.8 27b with DSH(DeepSeek Harness) is Amazing!! Experiences so far and perfomance.

Qwen 3.8 27b - Acquarium Panel Failure - Q8_K_XL Unsloth

Genie-style playable world model running 720p at 16 FPS on a single 5090 in 19GB VRAM

Paper claims RL for reasoning only changes 1-3% of tokens, and they replicate the gains without RL at ~1000x less compute

Qwen3.8 27B reasoning effort low/medium/xhigh comparison

I'm still running Qwen 3.5 122B. Should I switch to Qwen 3.8 27B?

Is ternary (1.58-bit) LLMs making a come back?

If you are at the lowest budget, which you can think of.Which hardware would you recommend to run? qwen 3.8 27b oWith like 50 tokens per second. I currently have a RTX 5070 Ti.

Anyone still use turboquant?

Qwen3.8-27B abliterated FP8: refusal 64–99% → 0–6%, and MMLU/GSM8K move less than 1.3 points

Anyone managed to get Qwen 3.8 27B running smoothly on vLLM? Can't get rid of endless thinking

5090: Windows or Linux for Qwen3.8.27b

How many people have 24gb over gpu here?

Show-off Saturday: Intel Arc B140 build.

Qwen3.8-27B vs Qwen3.6-27B writing ray-tracers in BASIC

Need assistance in getting my 4 AMD GPU setup running on a Z10PA-U8 mainboard

SOTA Apple Silicon Inference (August 15, 2026)

llama.cpp Windows Manager

The perfect way for Google to screw over OAI and Anthropic is by releasing a 120B dense multimodal Gemma model

club-5060ti refresh: tested RTX 5060 Ti presets, a proper high-context harness, and Qwen3.8 27B

Which Harness for Local Coding (Qwen 3.8 27b) do you Recommend?

Agentic harness for small models

RTX Pro 4500 vs. A100 (x4) - purchase being made today

The duality of man

US to tell partners they must pick sides in AI race with China

Best llm at understaning middle eastern languages? Arabic/Persian etc