KV cache as an agent runtime [R]

Our research team has been exploring an alternative approach to achieving interactivity and better responsiveness with LLM systems.

One of the team members wrote up a post about it:
https://research.yandex.com/blog/the-kv-cache-as-an-agent-runtime

The post sums up the overall idea of modifying models inference state (KV-cache) for achieving a more interactive LLMs. This idea was used in our lab's previous papers Hogwild! Inference, and AsyncReasoning, the post also contains a preview of the future work in this direction, where a Qwen3.8-27B agent is playing a DOOM env interactively using similar techniques.

We think that its interesting whether model inference/runtime design is itself an under-explored axis of agent capabilities, alongside models and the harness (e.g. harness is too abstract, changing model is too costly, do we need something in between?)