H2O-Lightning-4B: Apache-2.0 4B Decision model, official #1 open model on JevBench (above Jev)
Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.
Disclosure: I work at H2O.ai.
We released H2O-Lightning-4B, an open-weight (Apache-2.0) model for the "decisions API" style of inference that Jev made popular: you send a state plus typed questions (pick one / yes-no / score), and get calibrated probabilities back from a single forward pass. No generated tokens, so it's fast and cheap.
**Results (JevBench, public leaderboard):**
- Composite score 72.5, vs Jev 1.13 at 71.5; currently the top open model
- Leaderboard: https://benchmarkheaven.com/jev-models
**Running it:**
- Base: Qwen3.5-4B, fine-tuned
- Stock vLLM plus a small open shim (in the repo); ~30 ms per decision on an H100
- Your data stays local, no per-call fees
**Coming soon:** 12B and 31B versions, which in our internal testing are considerably smarter than
Jev, still open-weight and still one forward pass per decision.
**Demos** (inbox triage of 1,000 insurance claims, a multi-browser web agent, DOOM on the decision clock): https://youtu.be/2Qp04Wu0A14
Weights, model card and serving instructions: https://huggingface.co/h2oai/h2o-lightning-4b
Happy to answer questions about the setup and latency.
[link] [comments]
More from r/LocalLLaMA
-
The AI race: China's open-weight models trail US closed models by just 7 months
Oct 14
-
Anthropic Executive Says Open-Source AI Models Are Higher Risk
Oct 14
-
ComfyUI v0.40.0 released
Oct 14
-
So this is how they do it --> RTX 5090, once a consumer gaming card, is now on smugglers’ menu alongside cigarettes
Oct 14
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.