News / #music Tag Music 500 articles archived under #music · RSS Sign in to follow r/LocalLLaMA community 2mo ago MiniMax H3: A New Open-Weight Video Model, Live in ComfyUI MiniMax H3 is an open-weight, general-purpose multimodal video generation model that works across text, images, video, and audio. In ComfyUI, you can use H3 for text-to-video, image-to-video, first- and last-frame generation, and reference-driven creation. H3 jointly generates… 10 Hugging Face Daily Papers research 2mo ago Reinforcement Learning with Evolving Rubrics as Rewards for Audio Reasoning Abstract Audio reasoning is essential for machine understanding of the acoustic world. Reinforcement learning with verifiable rewards can elicit such reasoning, yet existing reward designs are complementary in their limitations: outcome-based rewards supervise only the final… 18 arXiv — NLP / Computation & Language research 2mo ago Separating Decision-Rule Misalignment from Readout-Coverage Limitations in Speech Language Models arXiv:2608.06409v1 Announce Type: new Abstract: Speech language models are increasingly evaluated on paralinguistic tasks by the accuracy of prompted answers, but answer accuracy combines failures at different stages of the audio-to-answer computation. We introduce a… 4 arXiv — NLP / Computation & Language research 2mo ago Do Audio Language Models Use Paralinguistic Evidence? Counterfactual Audits for Response Evaluation arXiv:2608.06718v1 Announce Type: new Abstract: Audio-language models (ALMs) are increasingly used as judges for speech-to-speech systems, but a judge that receives audio may not actually use paralinguistic evidence. We introduce counterfactual audits for paralinguistic response… 13 Hugging Face Daily Papers research 2mo ago StreamArena: Toward Continuous, Interactive, and Long-Horizon Agentic Streaming Video Understanding Abstract Deploying autonomous multimodal agents in continuous, real-world environments requires them to ingest unbounded audio-visual streams and maintain hour-scale memory. However, current evaluations predominantly rely on brief clips and multiple-choice formats. This design… 27 r/LocalLLaMA community 2mo ago The Gemma team will host a special event on August 20 Tweet by u/hackerllama Could be copium, but I would love to see Gemma 4.1 there with unified audio input for all model sizes perhaps even up to 120B, much improved tool calling (even with the latest template there are still bugs ), higher precision QAT from the start and… 20 r/LocalLLaMA community 2mo ago DeepSeek-V4-Flash-0731 Q8_K_XL sometimes stops mid-task in OpenCode - anyone else seeing this? Hey everyone, I've been experimenting with the new DeepSeek-V4-Flash-0731 release locally using the Unsloth Studio Q8_K_XL GGUF with OpenCode. Overall, it's been working really well, but I've noticed a strange behavior during longer agentic coding sessions. Once the context gets… 38 r/LocalLLaMA community 2mo ago Open-weight video gen that actually delivers. Five days with MiniMax H3 on local hardware. H3 weights went live on HuggingFace August 3rd and I started pulling them immediately. An omni-modal video model with native stereo audio in the same forward pass, where audio can actually drive the video generation? On open weights? I had to try it. Five days in, the quality is… 31 Simon Willison community 2mo ago Quoting John Gruber Me, I try to get into the mindset of playing live music, not recording a studio album. Except when I’m writing a piece where I really want it to be an album. Those aren’t rare , per se, but they’re occasional . If I tried to make every post a hall-of-famer I’d never get anything… 12 Simon Willison community 2mo ago Quoting John Gruber Me, I try to get into the mindset of playing live music, not recording a studio album. Except when I’m writing a piece where I really want it to be an album. Those aren’t rare , per se, but they’re occasional . If I tried to make every post a hall-of-famer I’d never get anything… 15 llama.cpp releases dev-tools 2mo ago b10326 tts: account for the vocoder pass in the timings line ( #26733 ) get_output runs the waveform work the pipeline defers to it, from a single trailing window to a full pass depending on the model. Measuring it keeps the reported total and the audio to process ratio honest.… 6 r/LocalLLaMA community 2mo ago parakeet.wgsl – Fast, accurate ASR in the browser, via raw WebGPU & SIMD WASM High-performance inference of NVIDIA's Parakeet TDT 0.6B V2 English transcription model, in the browser. Check out the live demo: https://parakeet.narcotic.sh/ A fully custom, dependancy-free implementation with raw WebGPU compute shaders and SIMD WebAssembly audio frontend. 1… 19 Simon Willison community 2mo ago The Tokenpocalypse Is Here: Companies Are Scrambling To Stop Spending So Much on AI The Tokenpocalypse Is Here: Companies Are Scrambling To Stop Spending So Much on AI There's a fun anecdote from Accenture (apparently via leaked meeting audio recordings) in this 404 Media piece from June 24th: “We’re seeing from some of the data internally at least that it’s… 5 Simon Willison community 2mo ago The Tokenpocalypse Is Here: Companies Are Scrambling To Stop Spending So Much on AI The Tokenpocalypse Is Here: Companies Are Scrambling To Stop Spending So Much on AI There's a fun anecdote from Accenture (apparently via leaked meeting audio recordings) in this 404 Media piece from June 24th: “We’re seeing from some of the data internally at least that it’s… 4 Hugging Face Daily Papers research 2mo ago Interpretable MEG Decoding of Perceived Speech: Cortical Sources and the Stimulus Features That Drive Retrieval Abstract Short segments of perceived speech can be retrieved from non-invasive magnetoencephalographic (MEG) recordings by deep networks trained with a CLIP-style objective against wav2vec 2.0 audio embeddings. Yet their weights do not map onto electrophysiological quantities,… 20 Ars Technica — AI news-outlet 2mo ago Suno hopes to go legit with watermarks for AI-generated music Suno plans watermarks and download limits to stop "large-scale abuse." 32 r/LocalLLaMA community 2mo ago nvidias nemotron omni only loads its text half on a mac, so i wrote the vision and audio towers in mlx nvidias nemotron omni is open weights and it sees, hears and reasons. theres already a 4bit mlx quant on hugging face but only the text backbone loads with standard mlx tooling. the model card says it plainly, the vision and audio towers need a runtime that implements the… 20 TechCrunch — AI news-outlet 2mo ago Amid legal battles, Suno says it will start watermarking songs Suno's watermarking feature comes as the company is fighting legal battles on several fronts. 35 r/MachineLearning community 2mo ago What are the biggest challenges in collecting high-quality speech and egocentric video datasets? [D] We're currently involved in collecting two types of datasets that seem to be increasingly important for multimodal AI Studio quality speech/audio datasets (high fidelity recordings) Egocentric household activity video datasets (first person daily task recordings) One thing that… 37 Hugging Face Daily Papers research 2mo ago AVE-Compass: Towards Holistic Evaluation for Audio-Video Editing Abilities Abstract While instruction-based video editing has advanced rapidly, real-world videos contain tightly coupled audio and visual signals, and editing one modality often requires coordinated changes in the other. Existing benchmarks primarily evaluate visual transformations on… 15 arXiv — NLP / Computation & Language research 2mo ago Spoken Function Calling: A New Perspective on Spoken Language Understanding for Large Audio Language Models arXiv:2608.05126v1 Announce Type: new Abstract: Spoken Language Understanding (SLU) is the core component of task-oriented dialogue systems and a pivotal link in achieving seamless human-agent interaction. While traditional SLU can effectively extract user semantics for… 7 r/LocalLLaMA community 2mo ago Scenema Audio Comes to ComfyUI, Runs on 8GB VRAM Hey everyone! Scenema Audio is now a native ComfyUI custom node. Same model that powers scenema.ai now quantized so it fits on 8GB VRAM. When we first released it a few months ago as an API and Docker stack, the full precision transformers were too heavy for most people to… 6 Hugging Face Daily Papers research 2mo ago Multi-Task Multi-Frame Visual Piano Transcription Abstract Audio-based piano transcription performs well on onset, pitch, and velocity, but the sustain pedal lets sound persist long after key release, so audio systems predict pedal-extended offsets rather than physical key release. Yet existing Visual Piano Transcription (VPT)… 38 r/LocalLLaMA community 2mo ago Building a Fully Local PDF Read-Aloud & PDF-to-Audiobook Desktop App with Kokoro 82M, Qwen, and llama.cpp Hey everyone, I’ve been building Speechfony - a desktop app for reading PDFs (and EPUBs) with offline text-to-speech. Open a document, listen sentence-by-sentence with highlighting, or export selected pages to an MP3. Everything runs locally: Kokoro for speech, and an on-device… 5 arXiv — NLP / Computation & Language research 2mo ago string2string Studio: An Interactive, In-Browser Platform for String-to-String Algorithms arXiv:2608.03984v1 Announce Type: new Abstract: We present string2string Studio, an interactive in-browser platform for string-to-string analysis across natural language processing, computational biology, and the digital humanities. The system integrates six main modules… 15 arXiv — NLP / Computation & Language research 2mo ago Reinforcement Learning with Evolving Rubrics as Rewards for Audio Reasoning arXiv:2608.02831v1 Announce Type: cross Abstract: Audio reasoning is essential for machine understanding of the acoustic world. Reinforcement learning with verifiable rewards can elicit such reasoning, yet existing reward designs are complementary in their limitations:… 14 Hugging Face Daily Papers research 2mo ago OmniPack: Unified Token Compression for Efficient Omni-modal Large Language Models Abstract Omni-modal large language models (Omni-LLMs) have achieved remarkable performance on audio-visual understanding tasks, but processing long and highly redundant visual and audio token sequences incurs substantial computational overhead, demanding aggressive token… 6 Hugging Face Daily Papers research 2mo ago AURORA-LM: Autoencoding Unified Representation for Continuous-Latent Diffusion Language Modeling Abstract Language remains an outlier in generative modeling: while images, video, and audio are increasingly modeled in continuous latent spaces, text generation still relies predominantly on discrete tokens. Existing continuous language models either inherit embedding spaces… 32 llama.cpp releases dev-tools 2mo ago b10274 mtmd: correcting duplicate empty audio chunks for short inputs ( #26536 ) correcting duplicate empty audio chunks for short inputs tests.sh code restored Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED… 30 TechCrunch — AI news-outlet 2mo ago Meet Wrinkles, an app that uncovers the hidden stories of the places around you Wrinkles, available on both iOS and Android, essentially acts as an AI-powered audio tour guide that reveals hidden history and local stories. 19 Simon Willison community 2mo ago PipeNetwork/minimax-h3-mlx PipeNetwork/minimax-h3-mlx MiniMax released MiniMax-H3 two days ago - they describe it as a "a general-purpose, omni-modal generative system", which in practice means it accepts text, images, audio and video and can use them to generate up to 15 second video clips with audio… 28 Simon Willison community 2mo ago PipeNetwork/minimax-h3-mlx PipeNetwork/minimax-h3-mlx MiniMax released MiniMax-H3 two days ago - they describe it as a "a general-purpose, omni-modal generative system", which in practice means it accepts text, images, audio and video and can use them to generate up to 15 second video clips with audio… 26 r/LocalLLaMA community 2mo ago Why are Chinese models better* at Frontend than the western top labs? I use A LOT both openAI and Anthropic products. When I need some frontend work (pure web dev) (or answer that feel less verbose and more to the point) I use Anthropic. For multimodality openAI feels better (understanding audio, screenshots, generating images, etc). But openAI… 17 r/LocalLLaMA community 2mo ago Time to finally migrate from LM Studio -> llama.cpp, your experience? Has anyone moved from LM Studio to llama.cpp? What was your experience like? What did you have to learn in order to recreate your experience? Which harness/GUI did you switch to? Thanks in advance!   submitted by   /u/CSEliot [link]   [comments] 19 r/LocalLLaMA community 2mo ago Is LM Studio abandoning their core product? Some of you may be aware that a few weeks ago, LM Studio announced a new agent, Bionic. This is pretty much an agentic harness for both local models and paid cloud models. But most aren't aware that LM Studio replaced almost every link to the original app that built their brand… 7 arXiv — Machine Learning research 2mo ago Interpretable MEG Decoding of Perceived Speech: Cortical Sources and the Stimulus Features That Drive Retrieval arXiv:2608.01481v1 Announce Type: new Abstract: Short segments of perceived speech can be retrieved from non-invasive magnetoencephalographic (MEG) recordings by deep networks trained with a CLIP-style objective against wav2vec 2.0 audio embeddings. Yet their weights do not map… 38 arXiv — NLP / Computation & Language research 2mo ago Discriminative Axis, Not Data Volume: What a Contrastive Corpus Teaches an Audio Embedding arXiv:2608.01560v1 Announce Type: new Abstract: Scaling the corpus is the default remedy when a contrastive representation lacks an attribute. We report a case where it does nothing, and identify what does: adding a lexical-speech round to a frozen-base multimodal embedding… 29 Hugging Face Daily Papers research 2mo ago SwanTale: Unified Multi-Speaker Speech and Audio Generation for Instruct and Zero-Shot Tasks Abstract Speech and audio generation is often needed in animation dubbing, audio drama, movies, advertising, games, podcasts, and short-video production. In these scenarios, creators may need to design voices without reference recordings, control speaker styles with natural… 4 Hacker News — AI on Front Page community 2mo ago MiniMax H3 Day-0 Support in ComfyUI: Open Weights, Native Audio, and 2K Video Article URL: https://blog.comfy.org/p/minimax-h3-day-0-support-in-comfyui Comments URL: https://news.ycombinator.com/item?id=49155629 Points: 202 # Comments: 58 18 arXiv — NLP / Computation & Language research 2mo ago TORUS: A Test of Rendering-Understanding Self-Coherence for Unified Audio Models arXiv:2607.28896v1 Announce Type: cross Abstract: Unified audio models capable of audio understanding, audio generation and, increasingly, audio editing are proliferating rapidly. Yet a basic question about them remains unanswered: do the two heads of a unified model agree about… 26 r/LocalLLaMA community 2mo ago MiniMax-H3 now on huggingface MiniMax H3 is a general-purpose, omni-modal generative system. It supports unified understanding of multimodal contexts composed of text, images, video, and audio, and can generate video with native stereo audio at resolutions up to 2K and durations of up to 15 seconds. Thanks… 16 r/LocalLLaMA community 2mo ago DeepSeek V4 @ IQ3XXS on M1 Ultra 128GB- 16 tok/s in LM Studio after patch M1 Ultra 128GB, Unsloth UD-IQ3_XXS, wired limit at 120GB. I was at 5-6 tok/s before the patch. Getting 15-16 tok/s now with the patched engine, and the output seems to have improved. Big thanks to this guy.   submitted by   /u/mil_phickelson [link]   [comments] 9 r/LocalLLaMA community 2mo ago DeepSeek V4 Flash 0731 IQ2_M benchmark for Dual 3060 and 96GB RAM ≈ 3.5 tok/s. Thanks to the community help I finally launched this llm. LM Studio refused to load weight onto second GPU but Unsloth Studio did so everything was done in there. Not a proper benchmark (used PC in parallel as well) but it gives an idea of the performance from dual 3060 with… 16 r/LocalLLaMA community 2mo ago EU AI Act takes effect tomorrow, August 2, 2026. 🤡 Basically you now have to mark all AI generated images, audio, video and text as AI generated. :P   submitted by   /u/xoxaxo [link]   [comments] 32 r/LocalLLaMA community 2mo ago Deepseek V4 Flash 0731. LM Studio loading only into RAM. The model refuses to load into VRAM and uses only RAM. What can be an issue? Q2_K_XL from Unsloth if that changes something.   submitted by   /u/esw123 [link]   [comments] 29 r/LocalLLaMA community 2mo ago [audio.cpp] Release 0.5: DramaBox expressive TTS, Confucius4 cross-lingual voice transfer, plus 7 more models and ROCm/HIP audio.cpp 0.5 is out :) The most fun new model in 0.5 is DramaBox . It is closer to prompt-directed voice acting. DramaBox is built on the LTX-2.3 audio architecture, and prompts can control emotion, delivery, laughs, sighs, pauses, transitions, and speaker behavior. Example… 15 Hugging Face Daily Papers research 2mo ago OmniScope: Modality-Decoupled Token Compression for Omnimodal Large Language Models Abstract Existing token compression methods for omnimodal large language models typically rely on one modality to determine what to retain in the other. We show that this assumption often breaks down: for the same query, audio and video relevance often peaks at different… 27 Hacker News — AI on Front Page community 2mo ago Getting 25 Gbps Thunderbolt Ethernet on My Mac Studio Article URL: https://www.jeffgeerling.com/blog/2026/getting-25g-ethernet-mac-thunderbolt/ Comments URL: https://news.ycombinator.com/item?id=49125034 Points: 200 # Comments: 100 24 r/LocalLLaMA community 2mo ago Minimax-H3 video model released, open weights coming in the next few days https://x.com/MiniMax_AI/status/2083006198828417501?s=20 Quote from their article: Today, we're launching MiniMax H3, a general-purpose multimodal generation model. H3 understands unified context across text, images, video, and audio, generating video with native stereo sound,… 12 arXiv — Machine Learning research 2mo ago Journey Operators for Structured Multi-Axis Composition arXiv:2607.26775v1 Announce Type: new Abstract: Many kinds of data have structure along one or more axes: words in a sentence, pixels in an image, nodes in a tree, frames in audio, or cells in a 3D volume. Along one axis, order matters: "the dog bit the man" is different from… 10 Page 6 of 10 · 500 articles ← Newer Older →
