News / #music Tag Music 500 articles archived under #music · RSS Sign in to follow arXiv — NLP / Computation & Language research 2mo ago Dissecting Sensitivity to Training Language in Self-Supervised Speech Learning Using Neural Audio Codec Tokens arXiv:2607.26350v1 Announce Type: cross Abstract: Neural audio codecs (NACs) have become popular for obtaining speech representations as discrete tokens. Beyond compression, discrete tokens can be used to train self-supervised learning (SSL) models. Such models, referred to as… 6 arXiv — NLP / Computation & Language research 2mo ago Prosody-driven Jailbreaks in Audio LLMs: A Controlled Study and Mechanistic Analysis arXiv:2607.26541v1 Announce Type: cross Abstract: Audio-capable foundation models enable end-to-end spoken interaction, but they also introduce safety risks beyond transcript content. It remains unclear how much jailbreak capability can arise from matched-text variation in… 29 Vercel — AI dev-tools 2mo ago Inkling Small from Thinking Machines is now available on AI Gateway Inkling Small from Thinking Machines is now available on AI Gateway. Inkling Small reaches performance comparable to the larger Inkling model at about a quarter of the size, using much less compute per task. It is a broad generalist with native reasoning over audio and images,… 30 Hugging Face Daily Papers research 2mo ago OmniDelta: Skill-Driven Budget Allocation for Token Compression in OmniLLMs Abstract Emerging Omni-modal Large Language Models (OmniLLMs) enable unified understanding of text, audio, and video, but their long audio-video token sequences introduce substantial memory and inference costs. Existing compression methods mainly focus on selecting important… 7 Vercel — AI dev-tools 2mo ago Grok Voice Think Fast 2.0 now available on AI Gateway Grok Voice Think Fast 2.0 from xAI is now available on AI Gateway. It is a speech-to-speech voice model that takes audio in and audio out, improving on the previous Grok Voice model in reasoning, transcription accuracy, and conversation. The model reasons in parallel with… 38 TechCrunch — AI news-outlet 2mo ago Fish Audio raises $50M seed to build AI voice models for creators and enterprises Since launching last year, the startup today has more than 8 million people using the open-source or hosted version of its models, and now generates annual recurring revenue of $21 million. 4 arXiv — NLP / Computation & Language research 2mo ago The JEPA Paradox in Language: The Geometry of Linguistic Alternatives arXiv:2607.23531v1 Announce Type: new Abstract: Joint-Embedding Predictive Architectures (JEPAs) are effective for images, video, and audio, yet deterministic JEPA-style latent prediction has not become a standard objective for text encoders. We argue that this gap reflects a… 37 arXiv — NLP / Computation & Language research 2mo ago Indic DiarBench: A Multilingual Joint Diarization and ASR Benchmark for Indian Languages arXiv:2607.23808v1 Announce Type: new Abstract: In this work, we introduce Indic DiarBench, a speaker diarization and ASR benchmark dataset spanning all 22 scheduled languages of India. This corpus comprises approximately 108 hours of natural multi-speaker audio from near-field… 5 arXiv — NLP / Computation & Language research 2mo ago Open Your Model's Eyes: Video and Context-Aware Multimodal Backchannel Prediction arXiv:2607.22729v1 Announce Type: cross Abstract: Backchannels, which signal listener states like empathy and understanding, are fundamental to natural human interaction. However, current approaches rely solely on audio and text. This omits crucial visual cues, such as facial… 18 Hugging Face Daily Papers research 2mo ago JarvisHub: An Open Harness for Canvas-Native Multimodal Creative Agents Abstract Creative AI is moving from single-step asset generation toward long-horizon multimodal production. Although recent generative models can synthesize high-quality images, videos, audio clips, UI elements, storyboards, slides, and other creative assets, real-world creative… 4 Hugging Face Daily Papers research 2mo ago OmniVAE: An Audio-Video VAE with Cross-Modal Alignment for Joint Generation Abstract Recent generative models are moving beyond silent video or standalone audio synthesis toward the joint generation of synchronized audio and video. Despite this progress, jointly generating audio and video with fine-grained cross-modal correspondence remains challenging… 12 r/LocalLLaMA community 2mo ago Viable ways to run K3 locally just curious how would people run it cheap if they really want kimi k3. dgx spark / strix halo clusters optane persistent memory platform + some gpus mac studio clusters orange pi 6 clusters ssd streaming + gpus multiple ddr3 + connectx 5 rdma clients two dgx stations power 10… 37 arXiv — Machine Learning research 2mo ago Phylogenetic signal in marine mammal and bird vocalizations captured by audio foundation models: the limited benefit of domain-specific pretraining arXiv:2607.22458v1 Announce Type: new Abstract: Do learned audio embeddings encode structure that nobody told them to encode? We probe four large pretrained audio models (AST, CLAP, BEATs-bio and BirdNET) with a downstream task none of them saw during training: recovering… 17 arXiv — Machine Learning research 2mo ago Probing Speaker Identity Sensitivity in Audio Deepfake Detectors arXiv:2607.21820v1 Announce Type: cross Abstract: Audio deepfake detectors are trained to distinguish genuine speech from synthetic speech and often perform well on standard benchmarks. Yet the same detector that achieves less than 1% error on one dataset can see its error rate… 31 r/LocalLLaMA community 2mo ago ai-sage/GigaChat3.1-Audio-10B-A1.8B · Hugging Face GigaChat Audio 10B is an audio-native LLM built on top of the GigaChat 3.1 Lightning text model. A Conformer speech encoder and a modality adapter feed audio embeddings directly into a Mixture-of-Experts decoder, so the model keeps the text quality of its base while adding… 36 r/LocalLLaMA community 2mo ago Benchmarks: TensorSharp vs. llama.cpp Cuda and Vulkan Benchmark: TensorSharp vs. llama.cpp I would like to share my latest open source local Unsloth (GGUF) LLM inference engine and applications. It supports many models from Unsloth, like Gemma4, DiffusionGemma, Qwen3.6 with multi-modal (image, vision, audio), Qwen… 38 r/LocalLLaMA community 2mo ago My GX10 died Everything ran fine, I was using UD 3.6 Q6 for 35 and 27B, each 4 concurrent requests at 200K context. I had Dify and Mastra to play around with, Unsloth studio to get around to and vLLM ready for whenever I decided to do some more testing. LLama-swap above lama.cpp and liteLLM… 30 r/LocalLLaMA community 2mo ago OrangePi AI Studio Pro - Qwen3.5-122B-A10B https://preview.redd.it/wbq8ullnbafh1.png?width=1409&format=png&auto=webp&s=e6d2fe2b1c87c724bc64003c25f917dcee53260f I finally got round to tweaking this, with a bit of help from GLM5.2. The trick to getting it running with vLLM (which I couldn't get anything really out of… 24 r/MachineLearning community 2mo ago I built an open-source multi-agent SDLC harness that beats a cold Claude Code run on large repos, by learning the repo once. Real benchmarks (incl. where it loses) inside. [P] Built an open-source AI coding agent that was 7%–75% cheaper than a cold "claude -p" run on 6/6 well-localized tasks across repositories up to ~82k LOC. The biggest difference: Cold agent: $6.83, 207 turns AutoDev Studio: ~$1.70 for the same bug The full benchmark (including… 14 r/LocalLLaMA community 2mo ago FLUX 3 - Real World Models: Towards Multimodal Flow Models as the Backbone of Visual Intelligence Introducing FLUX 3. One multi-modal model for Image, Video, Audio and Action-Prediction. Creations are truer to life in every kind of style. Blog Post : https://bfl.ai/blog/flux-3   submitted by   /u/pmttyji [link]   [comments] 23 arXiv — NLP / Computation & Language research 2mo ago An Evaluation Framework for Structured Audio Captions Validated by Controlled Perturbations arXiv:2607.21424v1 Announce Type: new Abstract: Recent advancements in automated audio captioning (AAC) have shifted from monolithic sentence generation toward structured formats that explicitly disentangle distinct acoustic and semantic properties. However, evaluating this… 8 r/LocalLLaMA community 2mo ago [audio.cpp] Release 0.4: Higgs Audio v3 TTS 4B (10x real time)+ Fish Audio S2 Pro in C++/GGML, full GGUF loading, Q8 speed and VRAM gains audio.cpp again :) Release 0.4 is out. The headline this time is new high-quality TTS coverage plus GGUF becoming a first-class across the project. What’s new: Added Higgs Audio v3 TTS 4B, Fish Audio S2 Pro, Voxtral Realtime ASR and two community models OuteTTS TTS and… 7 arXiv — NLP / Computation & Language research 2mo ago Abstraction Induces the Brain Alignment of Language and Speech Models arXiv:2602.04081v2 Announce Type: replace Abstract: Research has repeatedly demonstrated that intermediate hidden states extracted from large language models and speech audio models predict measured brain response to natural language stimuli. Yet, very little is known about the… 13 Hugging Face official-blog 2mo ago Bringing Nunchaku 4-bit Diffusion Inference to Diffusers Back to Articles a]:hidden"> Bringing Nunchaku 4-bit Diffusion Inference to Diffusers Published July 23, 2026 Update on GitHub Upvote 6 Pham Hong Vinh rootonchair guest Sayak Paul sayakpaul Large diffusion transformers can create stunning images (or even videos, audio snippets,… 22 arXiv — NLP / Computation & Language research 2mo ago Fusion Embedding: A Unified Embedding Space for Text, Image, Video, and Audio arXiv:2607.18666v1 Announce Type: new Abstract: A single embedding space that covers text, images, video, and audio lets one index serve every query a user can pose. Embedding models built on vision-language backbones now lead text/image/video retrieval benchmarks but lack audio… 36 arXiv — NLP / Computation & Language research 2mo ago From a Multilingual Streaming ASR Backbone to Kenyan-Language Systems: Data-Centric Adaptation of Nemotron 3.5 for Kikuyu, Dholuo, and Kalenjin arXiv:2607.18912v1 Announce Type: new Abstract: Automatic speech recognition (ASR) for African languages is constrained by orthographic inconsistency, annotation artifacts, missing audio, speaker and domain imbalance, and evaluation procedures that differ from deployment. We… 24 Vercel — AI dev-tools 2mo ago AI Gateway now supports streaming transcription AI Gateway now supports streaming transcription . Previously, transcription required a complete audio file and returned the full transcript in a single response. Now you can stream audio in as it's captured and get transcript updates back as the model produces them, keeping… 20 TechCrunch — AI news-outlet 2mo ago AI and the rise of the universal entertainment app Over the past decade, streaming platforms competed by dominating individual formats like music, video, podcasts, or audiobooks. Now, as AI makes it easier to create, organize, and recommend content, those distinctions are fading, pushing companies like Spotify, Netflix, YouTube,… 17 r/LocalLLaMA community 2mo ago Today I learnt the power of LocalLlama DISCLAIMER: No Ai was prompted in the creation of this post. Today I had an experience that complely blew my mind, I just had to write it down. As a bit of background I have been dabbling prompting local models using LM studio for the better part of 18 months now, keeping up to… 38 arXiv — NLP / Computation & Language research 2mo ago ESCUCHA: A Spanish Speech Benchmark for Heterogeneous Acoustic Conditions arXiv:2607.17812v1 Announce Type: new Abstract: As large audio language models (LALMs) advance, robust evaluation frameworks have become essential. In this context, Spanish speech understanding under realistic acoustic conditions has received particularly little attention. We… 6 arXiv — NLP / Computation & Language research 2mo ago Modeling turn-taking with distant viewing: investigating silence thresholds in human and AI-generated discourse arXiv:2607.18076v1 Announce Type: new Abstract: This study investigates silence gaps in two kinds of audiovisual material. We analysed thirty US situational comedies and fifty-one synthetic podcasts generated with Google NotebookLM. Gaps were compared across speaker gender,… 4 r/LocalLLaMA community 2mo ago Trellis.cpp now has a studio! When Trellis.cpp released, people were rightly complaining that while the port was nice, the usability barrier was still high since you had to navigate the command line and fetch all the weights manually. So now, Trellis.cpp has a built-in simple Studio binary: picks the proper… 19 r/LocalLLaMA community 2mo ago Introducing Scylla's Band, a new TTS model + inference framework with Android sample! Hey all! https://github.com/lowkeytea/scyllasband -> inference code https://huggingface.co/spybyscript/scyllasband -> model, LiteRT, ONNX, and voices https://lowkeytea.github.io/scyllasband/ -> sample audio for the voices, emotions, and languages. The tldr: 10 voices, 7… 13 Hugging Face Daily Papers research 2mo ago Qwen-Music Technical Report Abstract In this report, we introduce Qwen-Music, a powerful music generation model capable of producing highly musical and high-fidelity songs with complete vocal singing. Qwen-Music supports two core tasks: Text to Music Generation, which create entirely new songs from text… 9 arXiv — Machine Learning research 2mo ago AV-JEPA: Extending LeJEPA to Audio-Visual Self-Supervised Learning arXiv:2607.15295v1 Announce Type: cross Abstract: We present AV-JEPA, an elegant multimodal extension of LeJEPA to audio-visual self-supervised learning. Using an early-fusion Vision Transformer and modality dropout as masking, the model is trained to align the embeddings of… 31 arXiv — NLP / Computation & Language research 2mo ago Controlling Implicit Shortcut Reliance in L2 Spoken English Auto-markers arXiv:2607.16085v1 Announce Type: new Abstract: Increasingly, speech and language processing tasks take either audio or text directly rather than extracting features from these as the input to the classifier or regressor. Often these systems make use of complex, for example… 17 Hugging Face Daily Papers research 2mo ago Audio-Visual Flamingo: Open Audio-Visual Intelligence for Long and Complex Videos Abstract We present Audio-Visual Flamingo (AV-Flamingo), a fully open state-of-the-art audio-visual large language model (AV-LLM) for joint understanding and reasoning over audio, images, and long-form videos. Unlike prior AV-LLMs that primarily focus on short clips, AV-Flamingo… 9 r/LocalLLaMA community 2mo ago Hey Qwen Team: We Need a 100B MoE Model for Spark! Are there any Qwen team members here? Please release a 100B MoE model that I can run on Spark!   submitted by   /u/absurd-dream-studio [link]   [comments] 8 r/MachineLearning community 2mo ago TabFM Studio: point-and-click predictions on spreadsheets with tabular foundation models, fully local [P] I built a small web app that lets you run tabular foundation models (currently just Google's TabFM) on spreadsheets without writing any code. Just drop in a CSV/Excel file, click a column header to mark what to predict, hit predict. Rows where the target cell is filled become… 23 r/LocalLLaMA community 2mo ago Qwen and Gemma providers I like to use the MoE modles qwen3.6-35B and Gemma-4-26B. I noticed differences in result quality between versions from different providers, like bartowski, unsloth, lm-studio, google, etc. My tests dont give me a clear answer to though. Is there a rule if thumb which provider… 7 r/LocalLLaMA community 2mo ago Serving a fleet of Qwen3.5 122b sessions on a single Mac Studio (96GB) without losing your sanity Hello all Just following up on a post I made last week about my experiment to try minmax my Mac Studio. In particular, I've had quite a lot of success with pushing things even further. Across a 20 minute test with three concurrent sessions, my Mac Studio was offered 789,351… 5 Hugging Face Daily Papers research 2mo ago WanSong v1.0 Technical Report Abstract Music generation foundation models have recently attracted significant industry attention. However, achieving efficient generation and high-fidelity long-form audio while supporting controllability remains challenging. To address these needs, we present WanSong, a… 19 arXiv — NLP / Computation & Language research 2mo ago TikStance: A Multimodal and Hierarchical Dataset for Multi-target Stance Analysis in TikTok Political Conversations arXiv:2607.15240v1 Announce Type: new Abstract: Political discourse has increasingly moved to short-video platforms, yet computational analysis of such content remains constrained by the scarcity of datasets that jointly preserve audiovisual information and hierarchical… 15 Hugging Face Daily Papers research 2mo ago MultiRef-Compass: Towards Comprehensive Evaluation of Multi-Reference-to-Audio-Video Generation Abstract Multi-reference-to-audio-video (MR2AV) generation aims to generate coherent audio-video content conditioned on multiple references and textual instructions. Existing benchmarks mainly focus on text-driven generation, single-reference subject preservation, or isolated… 11 r/LocalLLaMA community 3mo ago Introducing LM Studio Bionic   submitted by   /u/pegasus912 [link]   [comments] 22 Hacker News — AI on Front Page community 3mo ago LM Studio Bionic: the AI agent for open models Article URL: https://lmstudio.ai/blog/introducing-lm-studio-bionic Comments URL: https://news.ycombinator.com/item?id=48939662 Points: 201 # Comments: 73 5 arXiv — Machine Learning research 3mo ago A Hybrid Mamba for Audio-Visual Navigation arXiv:2607.13110v1 Announce Type: new Abstract: Since the paradigm centered on convolutional neural networks and recurrent architectures was established in 2020, the fundamental backbone networks for audio-visual navigation have undergone no essential changes for more than five… 21 TechCrunch — AI news-outlet 3mo ago Hack suggests AI music generator Suno scraped YouTube for training data The hacker used an employee's credentials to access source code, which revealed how Suno scraped decades of audio. 29 Smol AI News news-outlet 3mo ago not much happened today **Thinking Machines Lab** launched **Inkling**, its first fully released open-weights foundation model family, featuring **975B parameters** with **41B active parameters** in a **Mixture-of-Experts** architecture. Inkling supports **multimodality** with text, image, and audio… 20 arXiv — Machine Learning research 3mo ago PolarBM: Complex-valued Boltzmann Machine for Modeling Audio Signals in Polar and Log-polar Coordinates arXiv:2607.12417v1 Announce Type: new Abstract: Although vast amounts of data, such as audio signal spectra, are naturally represented using complex numbers, conventional machine learning methods often simplify complex-domain problems by employing frameworks designed for… 4 Page 7 of 10 · 500 articles ← Newer Older →
