\n\t<a id=\"🏙️-urbanground-turns-a-real-scale-city-into-an-interactive-sandbox-for-multimodal-agents\" class=\"block pr-1.5 text-lg md:absolute md:p-1.5 md:opacity-0 md:group-hover:opacity-100 md:right-full\" href=\"#🏙️-urbanground-turns-a-real-scale-city-into-an-interactive-sandbox-for-multimodal-agents\" rel=\"nofollow\">\n\t\t<span class=\"header-link\"><svg class=\"text-gray-500 hover:text-black dark:hover:text-gray-200 w-4\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" aria-hidden=\"true\" role=\"img\" width=\"1em\" height=\"1em\" preserveAspectRatio=\"xMidYMid meet\" viewBox=\"0 0 256 256\"><path d=\"M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z\" fill=\"currentColor\"></path></svg></span>\n\t</a>\n\t<span>\n\t\t🏙️ UrbanGround turns a real-scale city into an interactive sandbox for multimodal agents.\n\t</span>\n</h1>\n<h2 class=\"relative group flex items-baseline\">\n\t<a id=\"what-can-you-do-with-urbanground\" class=\"block pr-1.5 text-lg md:absolute md:p-1.5 md:opacity-0 md:group-hover:opacity-100 md:right-full\" href=\"#what-can-you-do-with-urbanground\" rel=\"nofollow\">\n\t\t<span class=\"header-link\"><svg class=\"text-gray-500 hover:text-black dark:hover:text-gray-200 w-4\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" aria-hidden=\"true\" role=\"img\" width=\"1em\" height=\"1em\" preserveAspectRatio=\"xMidYMid meet\" viewBox=\"0 0 256 256\"><path d=\"M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z\" fill=\"currentColor\"></path></svg></span>\n\t</a>\n\t<span>\n\t\tWhat can you do with UrbanGround?\n\t</span>\n</h2>\n<ul>\n<li>Explore a georegistered 3D replica of Hong Kong from a first-person view</li>\n<li>Connect MLLM agents for closed-loop perception and physical control</li>\n<li>Evaluate visual grounding, navigation, exploration, planning, and adaptation</li>\n<li>Test agents under different times of day, weather, road closures, and moving pedestrians</li>\n<li>Build new embodied urban tasks on top of the sandbox</li>\n</ul>\n<h2 class=\"relative group flex items-baseline\">\n\t<a id=\"key-takeaways\" class=\"block pr-1.5 text-lg md:absolute md:p-1.5 md:opacity-0 md:group-hover:opacity-100 md:right-full\" href=\"#key-takeaways\" rel=\"nofollow\">\n\t\t<span class=\"header-link\"><svg class=\"text-gray-500 hover:text-black dark:hover:text-gray-200 w-4\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" aria-hidden=\"true\" role=\"img\" width=\"1em\" height=\"1em\" preserveAspectRatio=\"xMidYMid meet\" viewBox=\"0 0 256 256\"><path d=\"M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z\" fill=\"currentColor\"></path></svg></span>\n\t</a>\n\t<span>\n\t\tKey takeaways\n\t</span>\n</h2>\n<ul>\n<li>Current MLLMs are already strong at local urban perception, but this does not reliably translate into long-horizon spatial agency.</li>\n<li>As navigation becomes longer and more complex, small spatial errors accumulate and agents often fail to recover.</li>\n<li>Dynamic changes such as road closures and moving pedestrians remain particularly challenging.</li>\n</ul>\n<p>🌆 The sandbox is available as web and native builds, together with the evaluation code and tasks. We hope UrbanGround can serve as a playground for studying what it takes to turn strong local perception into reliable city-scale agency.</p>\n","updatedAt":"2026-08-28T02:40:08.562Z","author":{"_id":"64ba1a9d62c0bfc57398f1af","avatarUrl":"/avatars/0d416cacf4adfe13fcd281bd16d3e8eb.svg","fullname":"Tianjie","name":"jometeorie","type":"user","isPro":true,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":1,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9229943156242371},"editors":["jometeorie"],"editorAvatarUrls":["/avatars/0d416cacf4adfe13fcd281bd16d3e8eb.svg"],"reactions":[],"isReport":false}},{"id":"6a90f5fd54c0bb8fd060cf70","author":{"_id":"665db0a81d30854dbbc392df","avatarUrl":"/avatars/b898156df308c66b7e6a61c4ce5e8b53.svg","fullname":"dadong","name":"dadong1822","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false},"createdAt":"2026-08-28T02:44:13.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Up!","html":"<p>Up!</p>\n","updatedAt":"2026-08-28T02:44:13.229Z","author":{"_id":"665db0a81d30854dbbc392df","avatarUrl":"/avatars/b898156df308c66b7e6a61c4ce5e8b53.svg","fullname":"dadong","name":"dadong1822","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8139151930809021},"editors":["dadong1822"],"editorAvatarUrls":["/avatars/b898156df308c66b7e6a61c4ce5e8b53.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2608.27456","authors":[{"_id":"6a90ed86a64059bab69c35af","name":"Tianjie Ju","hidden":false},{"_id":"6a90ed86a64059bab69c35b0","name":"Zheng Wu","hidden":false},{"_id":"6a90ed86a64059bab69c35b1","name":"Yueqing Sun","hidden":false},{"_id":"6a90ed86a64059bab69c35b2","name":"Yuhan Cui","hidden":false},{"_id":"6a90ed86a64059bab69c35b3","name":"Bobo Li","hidden":false},{"_id":"6a90ed86a64059bab69c35b4","name":"Shengqiong Wu","hidden":false},{"_id":"6a90ed86a64059bab69c35b5","name":"Pengzhou Cheng","hidden":false},{"_id":"6a90ed86a64059bab69c35b6","name":"Haodong Zhao","hidden":false},{"_id":"6a90ed86a64059bab69c35b7","name":"Zongru Wu","hidden":false},{"_id":"6a90ed86a64059bab69c35b8","name":"Xinbei Ma","hidden":false},{"_id":"6a90ed86a64059bab69c35b9","name":"Doris Zhang","hidden":false},{"_id":"6a90ed86a64059bab69c35ba","name":"Kunling Li","hidden":false},{"_id":"6a90ed86a64059bab69c35bb","name":"Mong-Li Lee","hidden":false},{"_id":"6a90ed86a64059bab69c35bc","name":"Wynne Hsu","hidden":false},{"_id":"6a90ed86a64059bab69c35bd","name":"Hao Fei","hidden":false},{"_id":"6a90ed86a64059bab69c35be","name":"Qi Gu","hidden":false},{"_id":"6a90ed86a64059bab69c35bf","name":"Gongshen Liu","hidden":false},{"_id":"6a90ed86a64059bab69c35c0","name":"Zhuosheng Zhang","hidden":false}],"mediaUrls":["https://cdn-uploads.huggingface.co/production/uploads/64ba1a9d62c0bfc57398f1af/lFN5mEz9zXweU_sjtSVTo.mp4","https://cdn-uploads.huggingface.co/production/uploads/64ba1a9d62c0bfc57398f1af/_9HIQSam-d-6BPq1Pk55h.mp4","https://cdn-uploads.huggingface.co/production/uploads/64ba1a9d62c0bfc57398f1af/UuwxqV-A5AC-h68WApjjJ.mp4","https://cdn-uploads.huggingface.co/production/uploads/64ba1a9d62c0bfc57398f1af/e7vt2rhT0OFz6NyGb-whf.mp4","https://cdn-uploads.huggingface.co/production/uploads/64ba1a9d62c0bfc57398f1af/vM0MdwY2IHask860JPR4H.mp4","https://cdn-uploads.huggingface.co/production/uploads/64ba1a9d62c0bfc57398f1af/GOnX47kPGwANvqngKvxo_.mp4","https://cdn-uploads.huggingface.co/production/uploads/64ba1a9d62c0bfc57398f1af/w5aqcdIErSzqDoKR3HXmx.mp4","https://cdn-uploads.huggingface.co/production/uploads/64ba1a9d62c0bfc57398f1af/ToAKAyBWb0POfU-r4mMya.mp4","https://cdn-uploads.huggingface.co/production/uploads/64ba1a9d62c0bfc57398f1af/TKy6FrPoy6imafQhn83Uo.mp4"],"publishedAt":"2026-08-27T00:00:00.000Z","submittedOnDailyAt":"2026-08-28T00:00:00.000Z","title":"UrbanGround: From Local Perception to Spatial Agency in a Real-Scale City","submittedOnDailyBy":{"_id":"64ba1a9d62c0bfc57398f1af","avatarUrl":"/avatars/0d416cacf4adfe13fcd281bd16d3e8eb.svg","isPro":true,"fullname":"Tianjie","user":"jometeorie","type":"user","name":"jometeorie"},"summary":"Multimodal large language models (MLLMs) can interpret a street view, but urban agency depends on whether such local evidence remains useful after the agent starts to move. In this paper, we investigate how far current MLLM agents can turn local urban perception into reliable action in a complicated real-scale city. We propose UrbanGround, the first sandbox to make this question testable in a physically constrained replica of Hong Kong built from territory-wide 3D geospatial data. UrbanGround supports closed-loop interaction from a first-person view and provides an interactive map for navigation. Agents can directly enter the 3D city and explore from a first-person view. Our analysis follows the growth of the spatial problem through three research questions. We first test whether an agent can ground a local scene well enough to answer spatial questions after active observation. Then we ask whether that grounding supports navigation as destinations become farther away and less explicit. Finally, we examine whether the resulting behavior survives changes in route availability and pedestrian motion. Contemporary MLLM agents usually show useful atomic abilities in visual recognition and short-range spatial reasoning, while orientation and pedestrian-aware movement remain unreliable. Their central failure emerges over extended exploration, where local abilities do not compose into sustained goal-directed behavior and errors accumulate without effective correction. We hope UrbanGround will support broader study of how far current MLLM agents can explore reliably in complex, open-ended urban environments.","upvotes":37,"discussionId":"6a90ed87a64059bab69c35c1","projectPage":"https://urbanground.github.io","githubRepo":"https://github.com/UrbanGround/UrbanGround","githubRepoAddedBy":"user","ai_summary":"UrbanGround evaluates whether multimodal language model agents can sustain reliable navigation and spatial reasoning in a realistic 3D city replica, revealing that local perceptual skills fail to compose into extended goal-directed behavior.","ai_keywords":["multimodal large language models","MLLM agents","UrbanGround","3D geospatial data","closed-loop interaction","first-person view","spatial reasoning","pedestrian-aware movement","goal-directed behavior"],"ai_summary_model":"thinkingmachines/Inkling-Small","githubStars":17,"organization":{"_id":"63e5ef7bf2e9a8f22c515654","name":"SJTU","fullname":"Shanghai Jiao Tong University","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/1676013394657-63e5ee22b6a40bf941da0928.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"6816d98fc075e49c1b15928e","avatarUrl":"/avatars/6b24d047fc25075bedb3e74f78981bc0.svg","isPro":false,"fullname":"Tianjie Ju","user":"jometeorieNUS","type":"user"},{"_id":"6672b6a05f184b69364c1052","avatarUrl":"/avatars/a4f49a382bdc2fcfff3239a077d37b7e.svg","isPro":false,"fullname":"Haodong Zhao","user":"billhdzhao","type":"user"},{"_id":"665db0a81d30854dbbc392df","avatarUrl":"/avatars/b898156df308c66b7e6a61c4ce5e8b53.svg","isPro":false,"fullname":"dadong","user":"dadong1822","type":"user"},{"_id":"6491152545920777b90234ce","avatarUrl":"/avatars/80ab4f122bf731ddac8f141e924b8668.svg","isPro":false,"fullname":"张少卿","user":"zhangshaoqing","type":"user"},{"_id":"6776ae0c91b4c75dac91249c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6776ae0c91b4c75dac91249c/uJk3ZnRrzjPCcBNjmrWLI.png","isPro":false,"fullname":"Oran Feng","user":"xiachongfeng","type":"user"},{"_id":"63490424667da7e6737f49a4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/bpuWQVf5a-BFc2dsWnDPS.png","isPro":false,"fullname":"Li Bobo","user":"BradNLP","type":"user"},{"_id":"643910dbabdc6ce5351e4eb5","avatarUrl":"/avatars/92ec189cd4325b4d85fdfcd59f1ff1e3.svg","isPro":false,"fullname":"Yueqing Sun","user":"leqing","type":"user"},{"_id":"657998eee7a39f0aad20f197","avatarUrl":"/avatars/ccd3c28c89e1110391f52a4105872ae1.svg","isPro":false,"fullname":"Yu","user":"eden1114","type":"user"},{"_id":"65f5614f756646a946c71837","avatarUrl":"/avatars/b4277d3ffd4eb834fd768dc2a4150525.svg","isPro":false,"fullname":"dolabudaes","user":"dolabudaes","type":"user"},{"_id":"6a1fd9d481eee8267ebfe1f6","avatarUrl":"/avatars/4703ee72cb11285299f7fbc7128524a9.svg","isPro":false,"fullname":"Zhuowen Han","user":"Zhuowen02","type":"user"},{"_id":"66e02a3244a0a0d03316fea3","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/66e02a3244a0a0d03316fea3/jOtUgjO--4cetJdpvkWgm.jpeg","isPro":false,"fullname":"Yanlin Li","user":"yanlinli","type":"user"},{"_id":"6a910b281e927724ac70a07a","avatarUrl":"/avatars/0e98dc55b8582aa893f62578fcb1f517.svg","isPro":false,"fullname":"Chen Qingwen","user":"cqwhhhh","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"63e5ef7bf2e9a8f22c515654","name":"SJTU","fullname":"Shanghai Jiao Tong University","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/1676013394657-63e5ee22b6a40bf941da0928.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2608/2608.27456.md","query":{}}">

hf papers read 2608.27456

curl -LsSf https://hf.co/cli/install.sh | bash