arXiv — NLP / Computation & Language
500 articles archived · Visit source ↗ · RSS
-
arXiv — NLP / Computation & Language research 2d ago
Why RAGs Hallucinate: Penalty-Aware Evaluation of Retrieval-Augmented Generation Systems with Knowledge-Gap Canaries
arXiv:2608.26385v1 Announce Type: new Abstract: Volume-based accuracy rewards retrieval-augmented generation (RAG) systems for guessing: a system that answers everything outscores one that declines when its knowledge base cannot support an answer. Building on the…
6 -
Co-Evolving Structured Knowledge and Reasoning in Language Models
arXiv:2608.26386v1 Announce Type: new Abstract: Retrieval-augmented methods improve factual accuracy by grounding language models in external knowledge, but retrieving over unstructured text often introduces irrelevant context and offers limited control over the retrieved…
23 -
LowRankArena: A Standardized Evaluation Platform for SVD-Based LLM Compression
arXiv:2608.26389v1 Announce Type: new Abstract: SVD-based low-rank compression has become a fast-growing direction for reducing the memory and computational cost of large language models (LLMs). However, meaningful comparison across existing studies remains difficult as prior…
30 -
Case2Flow: Bridging Patient Cases and Guideline Flowcharts through Multimodal Retrieval
arXiv:2608.26414v1 Announce Type: new Abstract: Medical guidelines encode rich, evidence-based decision logic, yet the specific decision artifact a clinician needs is hard to locate within a guideline, let alone across guidelines covering plausible diseases and treatments. While…
26 -
AfriSwitch: A Benchmark for In-the-Wild African Code-Switched Speech Recognition
arXiv:2608.26434v1 Announce Type: new Abstract: Code-switching is pervasive in bilingual African conversation, yet most ASR systems assume monolingual input and are evaluated on curated monolingual benchmarks. We present AfriSwitch, a 61.36-hour human-transcribed benchmark of…
22 -
Vowel Signs Are Not Letters: A Pre-tokenization Ceiling on Multilingual Tokenizer Fertility
arXiv:2608.26449v1 Announce Type: new Abstract: Byte-level BPE tokenizers that use the HuggingFace ByteLevel pre-tokenizer inherit GPT-2's word regex, where a word is defined as \p{L}+, one or more Unicode letters. In abugida scripts, vowels are written as combining marks; this…
35 -
Compositional Generalization via Structural Identification in a Category-Theoretic Framework
arXiv:2608.26465v1 Announce Type: new Abstract: Compositional generalization is usually evaluated through model accuracy. We instead ask which structural or lexical identifications make held-out COGS examples admissible from the structures observed in training. Sentences are…
27 -
Sycophancy Suppression Can Impair Rational Updating: Anti-Sycophancy Should Preserve the Ability to Update
arXiv:2608.26511v1 Announce Type: new Abstract: Large language models often exhibit sycophancy, revising their answers to align with users when users push back. Such answer flips, however, can arise from different causes. One possibility is that the model simply aligns with the…
30 -
Multi-Expert Conformal Risk Control for Pairwise LLM Judging in Open-Ended Dialogue
arXiv:2608.26529v1 Announce Type: new Abstract: In this paper, we explore multi-expert Conformal Risk Control (CRC) algorithms for pairwise LLM-as-a-Judge evaluation in open-ended dialogue. Our core insight is that multi-expert aggregation offers a complementary remedy to CRC:…
11 -
SPEAR: Distilling Domain-Adaptive Reasoning Skeletons via Sequential Symbolic Alignment in Reinforcement Learning
arXiv:2608.26550v1 Announce Type: new Abstract: Reinforcement learning-based knowledge distillation has the potential to transfer complex reasoning from teacher to student models, yet it currently faces a critical dilemma: researchers must choose between sparse outcome-based…
34 -
SPT: Skills as Pre-Training Data for Agentic Language Models
arXiv:2608.26563v1 Announce Type: new Abstract: Agentic (tool-using) language models are mainly trained on tool-call traces and agent trajectories during post-training. These data provide direct behavioral supervision, but producing them requires task environments, execution,…
13 -
Dependency-Aware Revocable Decoding for Efficient Diffusion Large Language Model Inference
arXiv:2608.26574v1 Announce Type: new Abstract: Diffusion large language models (dLLMs) offer a promising alternative to autoregressive generation by decoding multiple tokens in parallel through iterative denoising. However, increasing decoding parallelism often degrades…
29 -
Double Trouble: Bilingual Pretraining Leaves Language-Conditioned Effects in Shared-Language Representations
arXiv:2608.26576v1 Announce Type: new Abstract: When researchers compare multilingual models for probing, interpretability, or cross-lingual transfer, they often align embedding spaces and assume that shared-language representations are comparable. We show that this assumption…
13 -
Surgical Alignment in Knowledge Graph Training for Clinical Diagnosis with Large Language Models
arXiv:2608.26587v1 Announce Type: new Abstract: Biomedical knowledge graphs (KGs) offer structured medical knowledge that can ground large language model (LLM) reasoning in clinical diagnosis application, yet how KG signal should be integrated into LLMs remains an open question.…
8 -
Benchmarking Clinical Decision Pathway Adherence in Large Language Models
arXiv:2608.26592v1 Announce Type: new Abstract: Following clinical decision pathways (CDPs) defined by clinical practice guidelines is essential for safe and reliable medical decision-making. However, existing medical large language model (LLM) benchmarks mainly evaluate…
4 -
Not Just Reason, Not Just Scan: Reinforcement Learning for Proactive Scientific Error Verification over Academic Paper
arXiv:2608.26596v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) are increasingly capable scientific assistants, yet they remain far from fully autonomous research. This transition requires models to actively inspect academic papers, build global evidence…
4 -
Which Metrics Save the Most Human Annotation? Prediction-Powered Evaluation and Meta-Evaluation
arXiv:2608.26638v1 Announce Type: new Abstract: Across various non-verifiable tasks, human evaluation is reliable but expensive, while automatic metrics are more scalable but often biased. Building on prediction-powered inference (PPI), we propose prediction-powered evaluation,…
35 -
Information-Guided Frontier Decoding: Contextual Utility-Driven Commitment in dMLLMs
arXiv:2608.26641v1 Announce Type: new Abstract: Decoding quality in diffusion multimodal language models (dMLLMs) depends heavily on the order in which masked tokens are committed. Existing confidence-based strategies prioritize locally easy tokens, but confidence does not…
19 -
Meta-Learning Where to Allocate Experts: Task-Conditioned Layer-Wise Compression for MoEs
arXiv:2608.26650v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) models route each token to a subset of expert networks, increasing capacity while keeping per-token computation sparse. In many deployed MoEs, the number of active experts is fixed across layers and tasks,…
16 -
Do LLMs Understand Personality? Rethinking Persona Fidelity Evaluation through Structured Behavioral Inference
arXiv:2608.26674v1 Announce Type: new Abstract: As large language models are increasingly deployed to simulate diverse human characters, ensuring persona fidelity, defined as the extent to which an agent's behavior consistently reflects the psychological and stylistic…
6 -
FOCUS & RePAIR: Mitigating Text Degeneration via Token-Level Guidance for Pruned Large Language Models
arXiv:2608.26676v1 Announce Type: new Abstract: Pruning is a practical approach to compress large language models (LLMs), but it can amplify text degeneration, especially repetition loops, even when perplexity and task accuracy remain largely unchanged. In this work, we present…
31 -
Beyond Reflection: Affirmation as a Promising Behavioral Marker Associated with Quality in Text-Based Counseling
arXiv:2608.26689v1 Announce Type: new Abstract: While AI-assisted text-based counseling is gaining attention, it remains empirically unclear which counselor behaviors are associated with higher dialogue quality. Existing research often focuses heavily on Reflection, borrowing…
32 -
Scaling phoneme-based TTS augmentation for ASR: A unified pipeline and controlled study
arXiv:2608.26697v1 Announce Type: new Abstract: Synthetic speech provides scalable supervision for automatic speech recognition (ASR), but its benefit depends on the selected texts, reference speech, and amount of synthesized data. We present a unified phoneme-based TTS-to-ASR…
14 -
PragAlign: Evidence-Sensitive Reply Assistance Across Chinese and Japanese Appropriateness Judgments
arXiv:2608.26700v1 Announce Type: new Abstract: Reply assistance in multilingual settings requires linguistic competence and culturally situated judgments of appropriateness. We present PragAlign, which separates context reading from selective clarification, and evaluate it…
23 -
Towards Expert Financial QA via Self-Improving RAG
arXiv:2608.26706v1 Announce Type: new Abstract: Expert-level financial question answering requires both grounded verification to catch numeric hallucinations and audit trails for regulatory compliance, attributes that standard single-pass RAG systems lack. We take a step toward…
6 -
Preserving General Capabilities during Domain Specialization with Uncertainty-Calibrated MOPD
arXiv:2608.26735v1 Announce Type: new Abstract: Specializing large language models to vertical domains improves domain-specific behavior but often degrades general capabilities such as reasoning, coding, instruction following, and creative writing. We study this domain--general…
29 -
Letters hide the truth from our eyes: English homophones have meaningfully different phonetic realizations
arXiv:2608.26749v1 Announce Type: new Abstract: The distribution of spoken word duration of English homophones is known to co-vary with frequency of use. This study investigates whether other aspects of the phonetic realization of homophones also differ. A series of quantitative…
26 -
Equal Ranking Quality, Different Decisions: Training Order-Consistent LLM Scorers
arXiv:2608.26762v1 Announce Type: new Abstract: Rerankers, reward models and multi-document QA scorers score candidate documents or responses in one LLM prompt, so each score depends on their order. Such scorers are selected on ranking quality, but their scores determine a…
27 -
Instruction Quality Matters: Refining Instructions for Effective Preference Learning
arXiv:2608.26779v1 Announce Type: new Abstract: Preference learning optimizes models using response pairs, yet the informativeness of these pairs is fundamentally shaped by the instructions from which they are generated. We identify instruction quality as a hidden bottleneck in…
25 -
Behavior2Trip: Towards Personalized Travel Planning via User Behavior Trajectory
arXiv:2608.26807v1 Announce Type: new Abstract: Travel planning agents assist users in generating personalized travel plans by modeling their individual preferences. Existing agents either rely on explicit user instructions or engage in multi-turn clarification to elicit user…
7 -
RuleWeaver: Benchmarking Rule-Centered Scenario Reasoning for Large Language Models
arXiv:2608.26832v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly applied to specialized domains, where effective use of domain expertise often requires reasoning over complex rules in concrete scenarios. However, existing benchmarks only partially…
5 -
Evaluating Confidence-Gated Retrieval with Matched Trajectory Replay
arXiv:2608.26846v1 Announce Type: new Abstract: Interactive language-model agents use confidence signals to decide whether to answer immediately, retrieve additional evidence (from memory or external knowledge), or defer. Yet confidence is usually evaluated in isolation, without…
13 -
Planting a Latent Variable in Natural-Looking Text: a More Realistic Test of Belief States in LLMs and Their Link to Concept Geometry
arXiv:2608.26887v1 Announce Type: new Abstract: LLMs are thought to track "belief states," i.e., running probability distributions over the latent variables that govern language (Shai et al., 2024; Sarfati et al., 2026), but so far this has only been comprehensively demonstrated…
6 -
TabuLM: Morphology-Aware Tabular Pre-training for Low-Resource Languages
arXiv:2608.26923v1 Announce Type: new Abstract: We present TabuLM, the first language model pre-trained on Kinyarwanda tabular data. Kinyarwanda is a morphologically rich Bantu language spoken by over 12 million people in Rwanda, yet lacks any dedicated tabular representation…
25 -
Mapping Written Words to Spoken Words in a Different Language Using Only Visual Grounding
arXiv:2608.26925v1 Announce Type: new Abstract: In many low-resource settings, even just eliciting speech for data collection is difficult. One promising approach has been to ask speakers to describe images. But how do we build models from such visually grounded speech data?…
12 -
KinyaEmbed: Contrastive Sentence Embeddings for Kinyarwanda via Multi-Stage Curriculum Training
arXiv:2608.26941v1 Announce Type: new Abstract: We present KinyaEmbed, the first dedicated sentence embedding model for Kinyarwanda, a morphologically rich Bantu language spoken by over 12 million people in Rwanda. Existing multilingual embedding models such as LaBSE, mE5-large,…
34 -
Squeezing More from Limited Data with Recursive Transformers
arXiv:2608.26973v1 Announce Type: new Abstract: Pre-training under limited data requires a different view of scaling than web-scale language modeling. With a fixed data budget but relatively abundant compute, increasing parameter count helps only up to an optimal scale; beyond…
27 -
JudgeStealer: Extracting LLM Judging Capabilities across Evaluation Protocols
arXiv:2608.26982v1 Announce Type: new Abstract: Large language model (LLM) judges are increasingly used across various evaluation scenarios, making their judgment capabilities valuable intellectual property. However, black-box access exposes these capabilities to model…
34 -
ITL: Interpretable Document Alignment with Structured Reference Frameworks
arXiv:2608.27031v1 Announce Type: new Abstract: Measuring alignment between documents and structured reference frameworks requires identifying conceptual evidence distributed throughout the text and reporting it through measures that are quantitative, interpretable, and…
19 -
Representing and Parsing Korean Constituency Structure at Different Levels of Granularity
arXiv:2608.27035v1 Announce Type: new Abstract: Korean constituency parsing raises a representational challenge because the terminal units of a phrase-structure tree do not straightforwardly correspond to simple surface words. Korean eojeols are morphologically complex spacing…
7 -
Reasoning about In-Context Samples for Machine-Translation
arXiv:2608.27036v1 Announce Type: new Abstract: Large Language Models (LLMs) can be trained to perform chain-of-thoughts reasoning in order to improve the reliability of their responses. In this work, we investigate how explicit reasoning can be leveraged for LLM-Based Machine…
18 -
Cascaded Batch Prompting
arXiv:2608.27038v1 Announce Type: new Abstract: Although batch prompting makes large language model inference more efficient by processing multiple instances simultaneously, it suffers from unpredictable downstream task performance. We propose cascaded batch prompting, a…
37 -
Research Design Tracking and Assessment for the Social Sciences
arXiv:2608.27049v1 Announce Type: new Abstract: Reliable assessment of causal research designs in the social sciences is critical for evidence-based policy-making, yet has so far relied entirely on manual expert analysis. We introduce Automated Research Design Tracking and…
32 -
DocTalkBN: A Novel Dataset of Expert Telemedicine Conversations in Bengali
arXiv:2608.27110v1 Announce Type: new Abstract: Reliable medical conversational AI requires authentic expert--patient interaction data, yet such datasets remain scarce, especially for low-resource languages such as Bengali. We present DocTalkBN, a large-scale multimodal dataset…
13 -
Cross-Lingual Alignment Without Joint Training: Do Monolingual Language Models Converge on Universal Representations?
arXiv:2608.27115v1 Announce Type: new Abstract: Cross-lingual alignment in multilingual language models is typically attributed to joint training: shared parameters, mixed-language batches, or explicit alignment objectives. We ask whether monolingual models trained on…
8 -
TwinKV: A Composable Repair Pass for KV Cache Eviction via Pairwise Key Redundancy
arXiv:2608.27128v1 Announce Type: new Abstract: Long-context inference is bottlenecked by the memory footprint of the key-value (KV) cache, especially for small models under tight resource budgets. Existing KV cache eviction methods score tokens using the model's attention…
14 -
Said Aloud, Read Different: Cross-Modal Instability in Multimodal Models
arXiv:2608.27135v1 Announce Type: new Abstract: Multimodal foundation models are increasingly used in speech-first assistants that must interpret spoken queries and produce visually grounded decisions. Yet it remains unclear whether semantically equivalent queries yield…
32 -
STAR : Sentence Translation Alignment Rate for Document-to-Document Machine Translation
arXiv:2608.27161v1 Announce Type: new Abstract: Large Language Models (LLMs) have enabled a shift from sentence-level to document-to-document (Doc2Doc) machine translation, promising improved global coherence. However, document-to-document generation in a single pass frequently…
11 -
Prediction of Prediction (PoP): Inter-Layer Activation Fusion for Single-Pass Hallucination Detection in Large Language Models
arXiv:2608.27165v1 Announce Type: new Abstract: Autoregressive large language models (LLMs) routinely generate factually incorrect outputs with high decoding confidence, limiting their deployment in high-stakes workflows. Existing output-stage uncertainty metrics can fail when…
8 -
When Text Misleads: Inconsistent-Aware Reasoning for Audio-Grounded Dialogue
arXiv:2608.27176v1 Announce Type: new Abstract: Understanding spoken dialogue requires joint reasoning over lexical content and paralinguistic acoustic signals such as emotion and conversational intent. However, existing evaluations often allow shortcuts based on transcripts or…
32