Hidden Reasoning from Claude and GPT are Decoded, and it is interesting

Yesteday a paper showed a gap that allows to see 100% of the reasoning tokens form ALL Claude and GPT models Stealing Reasoning Traces from Proprietary LLM APIs.

check it out, they have published lots of example reasonings.
this is very relevant for open soruce; for the following reason

- there is hint for benchmaxing; given a question form the benchmark AIME, Claude reasoning showed it KNOWS IT by heart and knows the answer; so yeah the plots we see for their performance beating the open source models -> could be overstated.

- when you are using the open source model and you see your model reasoning strangly; like strange words that does not make sense or OVERTHINKING. turns out this is very normal and even frontier models do it almost all the time.

- people now talk that this gap was used the whole time by China to distill frontier models and now that this gap is closed the distillation will be slowed down. let's see

so yeah, i believe open source is not soo far behind as it seems from the reasoning tokens, there is not secret sauce, it is just data, compute and engineering