I audited 443 GGUF quants across 25 repos. 64 of them can't be the quant their filename claims.

TL;DR: k-quants need tensor rows divisible by 256. When they aren't, llama-quantize quietly swaps in a ~4.5 bpw type and the file keeps its low-bit name. I audited 443 quants across 25 repos; 64 are affected. On Nemotron-3.5-Lightning all four IQ2 rungs are the same 4.58 bpw file under four different names. Tool and full census linked at the bottom.

Your quant's filename tells you what the quantizer was asked to make. It doesn't necessarily tell you what ended up in the file.

K-quants and i-quants need the first tensor dimension divisible by 256. When it isn't, llama-quantize substitutes a compatible 32-block type instead, often IQ4_NL for i-quants or Q4_0 for k-quants. Either way you can end up around 4.5 bits per weight instead of the low-bit type you requested. That's intentional, it's been in llama.cpp since PR #3747 in 2023, and the quantizer does print a warning. The catch is where the warning goes: into the quantize log. If you're downloading the finished GGUF you never see it. The filename still says IQ2_XXS, the model card says IQ2_XXS, and the metadata still describes an IQ2_XXS recipe.

https://preview.redd.it/mx32ahg9c6mh1.png?width=2320&format=png&auto=webp&s=b5183bfe1a1a109e202eb5800a1f4630f1d94e40

Above: every k/i-quant rung in bartowski's Nemotron-3.5-Lightning repo, claimed bpw vs measured. Two other makers uploaded the same model and got the same result, which is the first clue that this is the tooling rather than the uploader.

I wanted to know how far it spreads, so I wrote a tool that reads the tensor table and reports what's actually in the file. Works on a local GGUF or a whole HF repo. For remote repos it uses range requests to pull just the headers, usually a few MB, without downloading tensor data. One Python file, stdlib only, no pip install.

443 quants across 25 repos. The clearest affected cases:

- Nemotron-3.5-Lightning: n_embd is 2688 and the expert widths are 1856 and 3712, so about 99% of the parameters are forced into fallback types. All four IQ2 rungs are labeled between 2.06 and 2.56 bpw and all four measure 4.58. Four names, effectively the same density, across what looks like a 2.2x range.

- Qwen3.8-Flash-Next: 51.9% of parameters forced into fallback types. The file labeled UD-IQ1_S at 1.56 bpw measures 3.28.

- Nemotron-3-Super-120B: 18 of its 23 quant rungs contain fallbacks. That makes four affected repos in the Nemotron-H MoE family.

There were plenty of clean results too:

- MiniMax-M2.1: 23 rungs including a genuine IQ1_S, zero forced tensors.

- byteshape's Qwen3.6 quants: the filenames report measured bpw and my independent measurements match. Best labeling practice I found anywhere.

- bartowski's Ornith-1.5: a full 27-quant ladder, zero forced tensors.

- The dense Llama and Qwen controls came back clean too.

Every maker with an affected repo in my census also has a clean one using the same pipeline. The model's tensor dimensions decide this, not the maker.

That's why I don't think this is about careless uploaders or misleading model cards. The requested recipe is valid. The quantizer runs successfully. The fallback is intentional. Nothing in the finished file tells you that most of the recipe couldn't be applied.

Practical takeaway: on a fallback-dominated model, the lowest labeled rungs may not buy you the size reduction you think they do. If IQ2_XXS and IQ2_M land at the same measured density, there's not much reason to pick between them by filename, and you may be better off taking the honestly labeled Q4_0 or IQ4_NL and dropping the guesswork.

I'm not the first to notice the behavior. Issue #26616 asked for a --no-fallback option after someone got a 24.5 GB file where they expected about 18. What was missing was the measurement: how often it happens, which architectures it affects, and how much of each model is involved.

Background: the PR that added the fallback https://github.com/ggml-org/llama.cpp/pull/3747 and the open request for a fail-fast flag https://github.com/ggml-org/llama.cpp/issues/26616

Tool, census, and raw per-repo JSON: https://github.com/JoshBolding/ggufaudit

Point it at your own files and see what you actually have.

I've got a follow-up coming on what can be done about the affected models, because "just use the 4.5 bpw file" isn't a satisfying answer when the whole reason you wanted a low-bit quant was to fit a 16 GB card.

(Reposting this one, the formatting on my first attempt came out mangled and unreadable. Sorry to the two people who commented/saw it.)