Hugging Face Daily Papers · September 4, 2026 · 5 min read

Compile by Training: Turning Natural-Language Specifications into Local Neural Functions

Mirrored from Hugging Face Daily Papers for archival readability. Support the source by reading on the original site.

Like Read original ↗

What if LLMs built reusable tools instead of solving the same task over and over?\nProgram-as-Weights (PAW) compiles a natural-language function description into a small neural program that runs on a shared 0.6B local interpreter. Our original compiler does this in a single forward pass, so compilation takes only seconds.\nThis work, Compile by Training, adds a higher-accuracy mode: teacher models synthesize task-specific examples at compile time, then about a minute of finetuning specializes the program. On a hard subset of FuzzyBench where the fast PAW compiler did not produce exact-match outputs, Compile by Training reaches 83.6% semantic accuracy.\nBoth compilers produce reusable PAW programs that can be called locally like regular Python functions.\n<h3 class=\"relative group flex items-baseline\">\n\t<a id=\"examples\" class=\"block pr-1.5 text-lg md:absolute md:p-1.5 md:opacity-0 md:group-hover:opacity-100 md:right-full\" href=\"#examples\" rel=\"nofollow\">\n\t\t<svg class=\"text-gray-500 hover:text-black dark:hover:text-gray-200 w-4\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" aria-hidden=\"true\" role=\"img\" width=\"1em\" height=\"1em\" preserveAspectRatio=\"xMidYMid meet\" viewBox=\"0 0 256 256\"><path d=\"M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z\" fill=\"currentColor\"></path></svg>\n\t</a>\n\t\n\t\tExamples\n\t\n</h3>\n🗣️ Claudish — English ↔ Claude-style prose <a href=\"https://programasweights.com/claudish\" rel=\"nofollow\">https://programasweights.com/claudish</a>\n🤖 Avatar Director — natural language → action programs for a live 3D character <a href=\"https://programasweights.com/avatar\" rel=\"nofollow\">https://programasweights.com/avatar</a>\n🔒 PII Masker — detect and mask personal information locally <a href=\"https://programasweights.com/pii\" rel=\"nofollow\">https://programasweights.com/pii</a>\nTry the higher-accuracy compiler: <a href=\"https://programasweights.com/playground?compiler=paw-ft-bs48\" rel=\"nofollow\">https://programasweights.com/playground?compiler=paw-ft-bs48</a>\n","updatedAt":"2026-09-04T06:26:57.981Z","author":{"_id":"63081e15a670ed10f9d44229","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63081e15a670ed10f9d44229/w1b9uq-9774bMMgJbSPsS.jpeg","fullname":"Yuntian Deng","name":"yuntian-deng","type":"user","isPro":true,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":277,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8171238899230957},"editors":["yuntian-deng"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/63081e15a670ed10f9d44229/w1b9uq-9774bMMgJbSPsS.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2609.04199","authors":[{"_id":"6a9a61498f7c3b7557239686","name":"Yuntian Deng","hidden":false},{"_id":"6a9a61498f7c3b7557239687","name":"Pengyu Nie","hidden":false},{"_id":"6a9a61498f7c3b7557239688","name":"Stuart Shieber","hidden":false}],"mediaUrls":["https://cdn-uploads.huggingface.co/production/uploads/63081e15a670ed10f9d44229/WJTQ0-4Q9_U1t9NnrVvie.mp4"],"publishedAt":"2026-09-03T00:00:00.000Z","submittedOnDailyAt":"2026-09-04T00:00:00.000Z","title":"Compile by Training: Turning Natural-Language Specifications into Local Neural Functions","submittedOnDailyBy":{"_id":"63081e15a670ed10f9d44229","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63081e15a670ed10f9d44229/w1b9uq-9774bMMgJbSPsS.jpeg","isPro":true,"fullname":"Yuntian Deng","user":"yuntian-deng","type":"user","name":"yuntian-deng"},"summary":"Many recurring text functions are easy to describe but difficult to implement with rules, while calling a large remote model for every input introduces repeated cost, latency, and dependency on a provider. We present compile by training, which turns a natural-language specification into a reusable neural function. At compile time, teacher models generate task-specific examples that are used to train a small adapter for a compact interpreter. The resulting function runs without the teachers and can be stored, versioned, and composed like ordinary software. On FuzzyBench-Hard, a subset on which the Program-as-Weights fast compiler produced no exact matches, compile by training reaches 83.6% semantic accuracy. This higher accuracy comes with a higher compile-time cost: roughly a minute rather than seconds for the fast compiler. We deploy the compiler in a public interactive service and demonstrate compiled functions in a multi-site website helper, a language-controlled 3D avatar, and a bidirectional English-Claudish translator.","upvotes":29,"discussionId":"6a9a61498f7c3b7557239689","projectPage":"https://programasweights.com/playground?compiler=paw-ft-bs48","ai_summary":"Compile by training converts natural-language specifications into reusable neural functions by distilling teacher-generated examples into small adapters, enabling efficient deployment without remote model dependencies.","ai_keywords":["compile by training","neural function","adapter","compact interpreter","Program-as-Weights","semantic accuracy"],"ai_summary_model":"thinkingmachines/Inkling-Small","organization":{"_id":"6230c3ced93e84e2338765f3","name":"UWaterloo","fullname":"University of Waterloo","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/1647363004040-6230c34bacb94a81eec91ed5.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"63081e15a670ed10f9d44229","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63081e15a670ed10f9d44229/w1b9uq-9774bMMgJbSPsS.jpeg","isPro":true,"fullname":"Yuntian Deng","user":"yuntian-deng","type":"user"},{"_id":"66a017ead2b13e3f33d03725","avatarUrl":"/avatars/17c995158db3365f066f26cd4a3bbe29.svg","isPro":false,"fullname":"Boren Hong","user":"hongb2","type":"user"},{"_id":"6a669854fc9aae3cb525e879","avatarUrl":"/avatars/bff45e6282a5ab5039f85a49b61d6f1b.svg","isPro":false,"fullname":"Aaron Bishop","user":"aaronbishop22","type":"user"},{"_id":"6a7fa21baa0a49bf45659cd6","avatarUrl":"/avatars/5ea7f62935c8469f4f25f0ff8927f59f.svg","isPro":false,"fullname":"Abc Def","user":"abcdef2222","type":"user"},{"_id":"6a6d1b08bb661241e003a3f1","avatarUrl":"/avatars/4261e6c1cadc13df77114cba3619700c.svg","isPro":false,"fullname":"Abigail Morton","user":"abigailmorton","type":"user"},{"_id":"68f1a4e088e256c4ab477152","avatarUrl":"/avatars/e40704b1ebb1b78ccc9577121300793a.svg","isPro":false,"fullname":"Abram Durret","user":"abramdurret","type":"user"},{"_id":"6a669a442a2bb6b063be560f","avatarUrl":"/avatars/4f1f384bd1755af5414f39138c65361e.svg","isPro":false,"fullname":"Adam Grant","user":"adamgrant22","type":"user"},{"_id":"68e00dfa84ab145240d0725e","avatarUrl":"/avatars/ec9577fc780d3126f675729d70dd76a6.svg","isPro":false,"fullname":"Jianzhou Zhou","user":"jianzhouzhou","type":"user"},{"_id":"68e013db84ab145240d0d654","avatarUrl":"/avatars/8d067ecba36f5201db856c7d714ec32e.svg","isPro":false,"fullname":"Yuchen Wang","user":"yuchenwang2","type":"user"},{"_id":"6a4a411d96095a21aacbc54a","avatarUrl":"/avatars/19d10375491c149ab912eacc608e4708.svg","isPro":false,"fullname":"Cime","user":"come2222","type":"user"},{"_id":"654ed6406c46a91f120443ba","avatarUrl":"/avatars/4e0c2c7e56d648ed24c3b079acd237da.svg","isPro":false,"fullname":"Boren","user":"borent","type":"user"},{"_id":"6875be0edec586128138ef1a","avatarUrl":"/avatars/47a2eaacde2e3a5d3588743e919c74d4.svg","isPro":false,"fullname":"Cascade","user":"cascaded","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"6230c3ced93e84e2338765f3","name":"UWaterloo","fullname":"University of Waterloo","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/1647363004040-6230c34bacb94a81eec91ed5.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2609/2609.04199.md","query":{}}">

Papers

arxiv:2609.04199

Compile by Training: Turning Natural-Language Specifications into Local Neural Functions

Published on Sep 3

· Submitted by

Yuntian Deng on Sep 4

University of Waterloo

Upvote

Authors:

Abstract

Compile by training converts natural-language specifications into reusable neural functions by distilling teacher-generated examples into small adapters, enabling efficient deployment without remote model dependencies.

Generated by thinkingmachines/Inkling-Small

Many recurring text functions are easy to describe but difficult to implement with rules, while calling a large remote model for every input introduces repeated cost, latency, and dependency on a provider. We present compile by training, which turns a natural-language specification into a reusable neural function. At compile time, teacher models generate task-specific examples that are used to train a small adapter for a compact interpreter. The resulting function runs without the teachers and can be stored, versioned, and composed like ordinary software. On FuzzyBench-Hard, a subset on which the Program-as-Weights fast compiler produced no exact matches, compile by training reaches 83.6% semantic accuracy. This higher accuracy comes with a higher compile-time cost: roughly a minute rather than seconds for the fast compiler. We deploy the compiler in a public interactive service and demonstrate compiled functions in a multi-site website helper, a language-controlled 3D avatar, and a bidirectional English-Claudish translator.

View arXiv page View PDF Project page Add to collection

Community

yuntian-deng

Paper submitter about 3 hours ago

What if LLMs built reusable tools instead of solving the same task over and over?

Program-as-Weights (PAW) compiles a natural-language function description into a small neural program that runs on a shared 0.6B local interpreter. Our original compiler does this in a single forward pass, so compilation takes only seconds.

This work, Compile by Training, adds a higher-accuracy mode: teacher models synthesize task-specific examples at compile time, then about a minute of finetuning specializes the program. On a hard subset of FuzzyBench where the fast PAW compiler did not produce exact-match outputs, Compile by Training reaches 83.6% semantic accuracy.

Both compilers produce reusable PAW programs that can be called locally like regular Python functions.