Jev's calibration was measured. The LLMs won [D]
Mirrored from r/MachineLearning for archival readability. Support the source by reading on the original site.
| Source: Jev Benchmarks Its training method is literally called "Reinforcement Learning for Calibrated Decisions." Calibration gap vs human labels (lower = better): Yes/no: Jev 5.0, Gemini 3.8 Flash 2.0 Pick-one: Jev 9.8, DeepSeek V4.1 Flash 2.8 Rubric: Jev 19.7, GLM-5.3 12.9 It held to 95% accuracy, Jev still handles more decisions alone than any of them (86% of yes/no). Worse calibrated, better at knowing when it's right. [link] [comments] |
More from r/MachineLearning
-
I turned my dog Jackies Brain into a working AI [P]
Oct 14
-
Event2Vec: additive event embeddings where a linear score on a history decomposes exactly into per-event terms [P]
Oct 14
-
Binary noise classification using CRNN mean teacher algo [R]
Oct 14
-
Localized Neural learning vs backpropagation [D]
Oct 14
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.