Desh Raj @rdesh26

Speech + LLMs @nvidia | Previously: @Meta MSL, @jhuclsp, @IITGuwahati desh2608.github.io New York, NY Joined September 2009

Tweets

2K
Followers

5K
Following

2K
Likes

9K

Desh Raj @rdesh26

4 days ago

@sathwikmandava1 @ArtificialAnlys Qwen3.8-27B is dense model, meaning that it activates 27B param per token, vs 12B active for Super-3.5.

0 0 0 44 0

View Details

Meet NVIDIA Nemotron 3.5 Super VL Our new open model combines reasoning and tool use with text, image, document, and video understanding for long-running AI agents. 120B total parameters with 12B active 🧵

280 542 4K 568K 1K

View Details

Desh Raj @rdesh26

4 days ago

@harsh___d The easiest is to add a bunch of noise augmentation during training (you can check ESPNet recipes for how to do this). Having real noisy data would also be helpful, if expensive.

1 0 1 24 0

View Details

Desh Raj @rdesh26

5 days ago

I love full-duplex models and GPT-Live is awesome, but this claim is dubious at best. From Josh's post, most of the models hold up under "street noise" and "cafe chatter", but fail under "background TV", despite all of them being added at low energy levels. The only conclusion that we can draw from this is that most models are trained with noise augmentation resembling ambient noises or on in-the-wild data where it is unlikely to find TV noise. On the TV noise scenario itself, GPT-Live degraded by ~39% while Grok degraded only by 15%, despite not being a full-duplex model. From xAI's blog post: "We've focused on making Grok Voice Think Fast 2.0 perform exceptionally well in real-world settings, with substantial background noise and telephony compression." In short, they trained with a lot of real-world noise augmentation. Speech folks have worked on noise robustness and target-speaker recognition for a decade now, and came up with several cool architectural tricks. But if you look at ASR recipes from Kaldi/ESPNet, the only constant is noise augmentation and far-field simulation.

Peter Bakkum @pbbakkum

6 days ago

AFAIK gpt-live-1 is the only model here where turn detection is built into the audio model itself rather than separate, which is why it’s so much better perceiving turns in noisy environments. Enabled by the full duplex architecture

9 5 59 8K 24

7 5 59 4K 37

View Details

Desh Raj @rdesh26

5 days ago

I think it does well for the same reason that it does great on Tau-Voice: the post-training with delegation on enterprise agentic tasks is top notch. I get the point you're trying to make, and I agree that combining "when to respond" with "how to respond" is essential for a great voice agent, but I don't think this benchmark really captures that aspect. A more suitable benchmark would be something like this from @liva_ai: theliva.ai/blog/good-list…

1 0 5 162 2

View Details

Desh Raj @rdesh26

5 days ago

@jessearmand It's hard to say with certainty, but in general, if the frontier lab hasn't explicitly advertised "full-duplex" in the promotions, it is very unlikely that the model is FDX. Most voice models that advertise "real-time" are either cascades or half-duplex.

2 0 11 372 0

View Details

Desh Raj @rdesh26

5 days ago

@_josh_meyer_ I think they just used i-vectors extracted from the utterance itself, mainly as a way of speaker adaptation (rather than extraction). But the same idea has been used for TSE with SpeakerBeam (from NTT).

0 0 2 46 0

View Details

Desh Raj @rdesh26

5 days ago

In full disclosure, Kaldi recipes did append speaker i-vectors along with the frame-level features for the acoustic model, but this was mostly a remnant of the older speaker-discriminative training of HMM-GMM models. Later recipes (with E2E models, larger training data, and more noise augmentation) dropped these without any noticeable performance drop. Target-speaker recognition (or "device directedness" as it is called in industry) is a real problem when the energy level of the interfering speaker is similar to the main speaker, but again sonething can be reasonably tackled through appropriate augmentation strategies.

2 0 5 228 0

View Details

Desh Raj @rdesh26

5 days ago

@_josh_meyer_

0 0 2 47 0

View Details

Josh Meyer @_josh_meyer_

6 days ago

How to kill a voice agent? Just turn on the TV ☠️ We benchmarked 14 voice agents with a TV playing in the background. Task completion fell from 38.7% to 8.6% 🧵/

29 19 189 32K 108

View Details

Desh Raj @rdesh26

5 days ago

@_josh_meyer_ Ahh time to dig up the old papers on noise-robust and target-speaker ASR!

1 0 3 223 0

View Details

Desh Raj @rdesh26

6 days ago

@rissa_cao Could you link the model weights as well, thanks!

1 0 1 289 0

View Details

Desh Raj @rdesh26

6 days ago

@FishAudio I didn't find any links for the model weights?

1 0 1 75 0

View Details

Joma Tech @jomatech

a week ago

Introducing ChatGPT for Dishwashing GPT-6 achieves state-of-the-art results on dishwashing benchmarks for long-running jobs and difficult stains. The Navier–Stokes Millennium Prize Problem is a $1 million challenge about whether the equations describing fluid motion can break down. In this video, I accidentally found a counter-example while trying to optimize my dishwasher. But in reality, OpenAI used 10,000 AI agents to find a proof in 88 hours: openai.com/index/navier-s… Parody of the original GPT-6 Astra ad @OpenAI #chatgpt_Partner #chatgpt Credits: Written and Directed by Joma Co-Writer - Henry Connor Coan IV @coan_henry Production by Dauntlus Studios @dauntlus.studios Producer/Co-Director - Marcus Liew @dauntlus Camera Operator - Amos Lee @amoselijahlee Art Director - Alethea Soo @aletheasoo Art Assistant - Alicia Lim @aliciatkl_ Assistant Editor - GPT-6 Astra Music: Tensions Run High by Soundridemusic youtube.com/watch?v=Ly9H63…

423 960 11K 1.1M 3K

View Details

Allan @niemerg

a week ago

I think I'm going to get a lot of mileage out of this one.

Andreas Kirsch 🇺🇦 @BlackHC

2 weeks ago

Mathematicians are coping so hard they might abolish irrational numbers again and turn the field into a guild terrytao.wordpress.com/2026/10/07/ahm…

56 46 1K 711K 291

72 254 7K 628K 1K

View Details

Desh Raj @rdesh26

2 weeks ago

@voicearena_ai @smallest_AI If you are ranking diarization+ASR systems, why are you using DER for ranking? There are several metrics for speaker-attributed transcription such as cpWER and tcpWER. You can use MeetEval (github.com/fgnt/meeteval) for scoring.

0 0 1 115 1

View Details

tuk’s ˚⋆ @lilieslores

2 weeks ago

love how in 90s and 2000s romcoms, nyc was never just a backdrop but a character of its own

104 5K 46K 1.4M 5K

View Details

Desh Raj @rdesh26

2 weeks ago

@shinjiw_at_cmu Congratulations Shinji! Huge milestone, 200 coming soon :)

0 0 0 127 0

View Details

Desh Raj @rdesh26

2 weeks ago

this was not in my bingo card for 2026

Ben Burtenshaw @ben_burtenshaw

2 weeks ago

PSA: @mervenoyann is destined to teach @pewdiepie about local agents and post-training. upvote her plea and make a dream come true. github.com/odysseus-dev/o…