EMNLP 2026; code is available at <a href=\"https://github.com/seokwon99/Multi3IR\" rel=\"nofollow\">https://github.com/seokwon99/Multi3IR</a>.</p>\n","updatedAt":"2026-09-03T02:06:20.663Z","author":{"_id":"651ea954e69540c396f64d98","avatarUrl":"/avatars/0ade6e2a86863f6cee4e1423be23dbbd.svg","fullname":"seokwon, song","name":"seokwon99","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":2,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8721494674682617},"editors":["seokwon99"],"editorAvatarUrls":["/avatars/0ade6e2a86863f6cee4e1423be23dbbd.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2608.30949","authors":[{"_id":"6a9843e7fea818274321fcc9","user":{"_id":"651ea954e69540c396f64d98","avatarUrl":"/avatars/0ade6e2a86863f6cee4e1423be23dbbd.svg","isPro":false,"fullname":"seokwon, song","user":"seokwon99","type":"user","name":"seokwon99"},"name":"Seokwon Song","status":"claimed_verified","statusLastChangedAt":"2026-09-02T16:45:04.885Z","hidden":false},{"_id":"6a9843e7fea818274321fcca","name":"Sohyeon Kim","hidden":false},{"_id":"6a9843e7fea818274321fccb","name":"Gunhee Kim","hidden":false}],"publishedAt":"2026-08-31T00:00:00.000Z","submittedOnDailyAt":"2026-09-03T00:00:00.000Z","title":"MULTI3IR: A Benchmark for Multi-perspective Multi-domain Multi-modal Information Retrieval","submittedOnDailyBy":{"_id":"651ea954e69540c396f64d98","avatarUrl":"/avatars/0ade6e2a86863f6cee4e1423be23dbbd.svg","isPro":false,"fullname":"seokwon, song","user":"seokwon99","type":"user","name":"seokwon99"},"summary":"Information retrieval (IR) increasingly targets open-ended queries that admit diverse perspectives. Existing IR benchmarks, however, focus primarily on closed-ended queries, while even open-ended benchmarks largely consist of queries whose supporting documents span a single subject domain and modality. We introduce Multi^3IR, a benchmark that evaluates how well retrievers cover the multifaceted perspectives of open-ended queries across diverse domains and modalities. It comprises 104.9K Stack Exchange queries, each annotated with perspective descriptions that capture the query's implicit viewpoints. We further propose SPIN, a parameter- and label-efficient method that learns noise vectors to steer embeddings toward diverse yet meaningful semantic directions. Experiments show that existing multimodal retrievers suffer from single-perspective bias, while SPIN substantially improves perspective coverage on Multi^3IR and generalizes well to unseen open-ended IR benchmarks. The dataset and experimental code are available at https://github.com/seokwon99/Multi3IR.","upvotes":6,"discussionId":"6a9843e8fea818274321fccc","ai_summary":"A new benchmark and steering method improve retrieval coverage of diverse perspectives across domains and modalities for open-ended queries.","ai_keywords":["open-ended queries","multimodal retrievers","single-perspective bias","noise vectors","embedding steering","perspective coverage","Multi3IR"],"ai_summary_model":"thinkingmachines/Inkling-Small","organization":{"_id":"66d54dc8033492801db2bf5a","name":"SeoulNatlUniv","fullname":"Seoul National University","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/659ccc9d18897eb6594e897f/_-0BM-1UyM-d-lRiahFnf.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"6a2da6c8ca070ee12c6e396c","avatarUrl":"/avatars/0355287dcabaa67dbc7f0b10b87451f9.svg","isPro":false,"fullname":"Joe Mama","user":"JoeMama123123123","type":"user"},{"_id":"651ea954e69540c396f64d98","avatarUrl":"/avatars/0ade6e2a86863f6cee4e1423be23dbbd.svg","isPro":false,"fullname":"seokwon, song","user":"seokwon99","type":"user"},{"_id":"63ac5701c21e60a3e9b58aa7","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63ac5701c21e60a3e9b58aa7/g6EX7diOpuA94R2ab-rZC.png","isPro":true,"fullname":"Dipankar Sarkar","user":"dipankarsarkar","type":"user"},{"_id":"699eaa14ef8843d6e00c3f79","avatarUrl":"/avatars/0836ccd24855ec87bffe4eff862c4a05.svg","isPro":false,"fullname":"Liam Nguyen","user":"liamnguyen79","type":"user"},{"_id":"6984de1f93351a77acaa7ac3","avatarUrl":"/avatars/b172c98f1c7c9ce15be299e41d9c442a.svg","isPro":false,"fullname":"Laura Becker","user":"bossypike69","type":"user"},{"_id":"66bc565ebfa218d3b1650e4d","avatarUrl":"/avatars/9c2bedfca245ddd8f8e33f3b780035e6.svg","isPro":false,"fullname":"Yunah Jang","user":"YunahJang","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"66d54dc8033492801db2bf5a","name":"SeoulNatlUniv","fullname":"Seoul National University","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/659ccc9d18897eb6594e897f/_-0BM-1UyM-d-lRiahFnf.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2608/2608.30949.md","query":{}}">
MULTI3IR: A Benchmark for Multi-perspective Multi-domain Multi-modal Information Retrieval
Abstract
A new benchmark and steering method improve retrieval coverage of diverse perspectives across domains and modalities for open-ended queries.
Information retrieval (IR) increasingly targets open-ended queries that admit diverse perspectives. Existing IR benchmarks, however, focus primarily on closed-ended queries, while even open-ended benchmarks largely consist of queries whose supporting documents span a single subject domain and modality. We introduce Multi^3IR, a benchmark that evaluates how well retrievers cover the multifaceted perspectives of open-ended queries across diverse domains and modalities. It comprises 104.9K Stack Exchange queries, each annotated with perspective descriptions that capture the query's implicit viewpoints. We further propose SPIN, a parameter- and label-efficient method that learns noise vectors to steer embeddings toward diverse yet meaningful semantic directions. Experiments show that existing multimodal retrievers suffer from single-perspective bias, while SPIN substantially improves perspective coverage on Multi^3IR and generalizes well to unseen open-ended IR benchmarks. The dataset and experimental code are available at https://github.com/seokwon99/Multi3IR.
Community
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2608.30949 in a model README.md to link it from this page.
Cite arxiv.org/abs/2608.30949 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2608.30949 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.