How far are we from a local model being able to optimize it's own runtime stack?
Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.
A lot of projects have come up recently consisting of specific hardware config specialized forks of existing conferencing engines, but they all seem to be made using Claude or OpenAI's biggest frontier models.
So given that open source is about 4-6 months behind, can we reasonably expect that local runnable models will start having the ability to dissect a big repo like llama cpp and optimize it for a given set of hardware sometime starting next year?
Or maybe are models like deepseek4.1 or glm5.3 already able to do that? What about their smaller twins? v4fvision and 5.3flash? Maybe even qwen flash next is capable of doing this?
Qwen 4 is coming out end of October, anyone expect that model to be able to pull self-optimization off?
[link] [comments]
More from r/LocalLLaMA
-
The AI race: China's open-weight models trail US closed models by just 7 months
Oct 14
-
Anthropic Executive Says Open-Source AI Models Are Higher Risk
Oct 14
-
ComfyUI v0.40.0 released
Oct 14
-
So this is how they do it --> RTX 5090, once a consumer gaming card, is now on smugglers’ menu alongside cigarettes
Oct 14
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.