BREAKING: world's first culinary intelligence benchmark for LLMs now on arXiv.</p>\n<p>We tested the strongest model from 16 labs across 534 executable culinary tasks powered by Epicure.</p>\n<p>Grok 4.6 leads the pack.</p>\n","updatedAt":"2026-08-24T11:02:54.097Z","author":{"_id":"64442f46af034cdfd69d5bc4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64442f46af034cdfd69d5bc4/xBD3PKu6sOMAKVsvi4hQ6.jpeg","fullname":"Josef Chen","name":"josefchen","type":"user","isPro":true,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":4,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9024754166603088},"editors":["josefchen"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/64442f46af034cdfd69d5bc4/xBD3PKu6sOMAKVsvi4hQ6.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2608.20574","authors":[{"_id":"6a8c1bc13d26296ea3091b6a","user":{"_id":"64442f46af034cdfd69d5bc4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64442f46af034cdfd69d5bc4/xBD3PKu6sOMAKVsvi4hQ6.jpeg","isPro":true,"fullname":"Josef Chen","user":"josefchen","type":"user","name":"josefchen"},"name":"Josef Chen","status":"claimed_verified","statusLastChangedAt":"2026-08-24T14:36:51.966Z","hidden":false},{"_id":"6a8c1bc13d26296ea3091b6b","name":"Erim Hayretci","hidden":false}],"mediaUrls":["https://cdn-uploads.huggingface.co/production/uploads/64442f46af034cdfd69d5bc4/0nM6g64I_SrJZP0KkXwjc.png"],"publishedAt":"2026-08-20T00:00:00.000Z","submittedOnDailyAt":"2026-08-24T00:00:00.000Z","title":"FlavourBench: Ranking Frontier Language Models with Executable Culinary Ground Truth","submittedOnDailyBy":{"_id":"64442f46af034cdfd69d5bc4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64442f46af034cdfd69d5bc4/xBD3PKu6sOMAKVsvi4hQ6.jpeg","isPro":true,"fullname":"Josef Chen","user":"josefchen","type":"user","name":"josefchen"},"summary":"Open-ended language-model benchmarks usually inherit a judge: a human preference panel, another model, or a brittle exact-match key. We introduce FlavourBench, an automated benchmark in which a versioned culinary system supplies dense, executable ground truth. Each task presents eight ingredients and asks for a three-ingredient portfolio; before model execution, Epicure scores all 56 possible portfolios. We evaluate 27 frontier endpoints on an identical 534-task core spanning substitution, pairing, and constrained composition. Every ranked model has exactly 89 valid responses per panel and family (14,418 model-task cells total), eliminating differential missingness from the leaderboard. The FlavourBench Score is the equal-family mean of the frozen task scores. We use 50,000 anchor-cluster bootstrap replicates for simultaneous 95% score bands and 100,000 sign-flip draws for all 351 paired model contrasts, with Holm control. The two independently compiled panels correlate at r = 0.89 (rank rho = 0.80). Grok 4.6 has the largest point estimate at 65.1 (simultaneous 95% CI 61.0-69.2); 101 of 351 model pairs are resolved. The release includes the prompts, all portfolio score maps, raw responses, exact routes, content hashes, and an offline verifier that reconstructs every result.","upvotes":1,"discussionId":"6a8c1bc13d26296ea3091b6c","projectPage":"https://huggingface.co/spaces/josefchen/flavourbench","githubRepo":"https://github.com/josefchen/flavourbench","githubRepoAddedBy":"user","ai_summary":"FlavourBench evaluates language models on culinary portfolio tasks using executable ground truth, statistical rigor, and fully reproducible verification.","ai_keywords":["FlavourBench","executable ground truth","portfolio scoring","bootstrap replicates","Holm control","simultaneous confidence intervals","paired model contrasts","offline verifier"],"ai_summary_model":"thinkingmachines/Inkling-Small","githubStars":4},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"64442f46af034cdfd69d5bc4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64442f46af034cdfd69d5bc4/xBD3PKu6sOMAKVsvi4hQ6.jpeg","isPro":true,"fullname":"Josef Chen","user":"josefchen","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2608/2608.20574.md","query":{}}">

Get this paper in your agent:

Don't have the latest CLI?

Cite arxiv.org/abs/2608.20574 in a model README.md to link it from this page.

Add this paper to a collection to link it from this page.