mikail @Gradientdinner

Research Scientist @nvidia 🌁 | PhD @MIT San Francisco, CA Joined January 2019

Tweets

2K
Followers

3K
Following

2K
Likes

28K

Dimitris Papailiopoulos @DimitrisPapail

a week ago

It sounds insane, because we're used to a particular way of doing research: of giving and receiving credit, of counting beans, of hiring, recruiting, and promoting. And the machine god came to crush that game. The old game is dead. That reasonably causes incredible amounts of dread for many folks, who understand this as the death of research. But it's not about Research, but the way we played the game. The old game was built around scarcity, scarce compute, scarce students, scarce hours, scarce people who knew the right technique and method etc. Most of a research career went into managing and playing along these constraints. Those walls just came crushing down. One person with a good question can now do what used to take a lab and five years. You do not need permission, you do not need a big grant, you do not need a professor or researcher title, or a team of twenty experts , or one thousand gpus to stare straight into the first light of the universe. How absolutely incredible

Dimitris Papailiopoulos @DimitrisPapail

a week ago

I know this may sound insane, but this is the best time to be a Researcher.

73 267 2K 219K 448

59 183 1K 81K 409

View Details

mikail @Gradientdinner

2 weeks ago

@wen_kaiyue @_arohan_

0 0 2 192 0

View Details

Maximus Spritius @MozeTech

2 weeks ago

I just merged two of the highest grossing games ($5 billion+) in history using fable 5.5 (rerouting) in just one shot! I brought Minecraft to the Pokemon world, you can now craft, build and catch pokemon all in the iconic pallet town. There's no longer any reason for anyone not to be making their own billion dollar game idea, it's literally one click away now. With everyone now having the ability to bring their imaginations to life, expect to see a whole lot more games just like this. We're witnessing the dawn of incredible crossovers across the gaming world! Try it out here: pokemon-redstone.vercel.app

376 559 10K 1.7M 4K

View Details

Soufiane Hayou @hayou_soufiane

a month ago

@xidulu I think one possible implication is that Muon learns more robust representations. Not sure though.

3 1 12 2K 6

View Details

lyra @_lyraaaa_

a month ago

the fly brain can play beat saber

1K 8K 86K 23.1M 17K

View Details

mikail @Gradientdinner

2 months ago

@borisdayma @evaninwords @HessianFree @pcuenq @eraznafre @GoogleAI Except the blocked implementation of course

0 0 2 54 0

View Details

mikail @Gradientdinner

2 months ago

@borisdayma @evaninwords @HessianFree @pcuenq @eraznafre @GoogleAI github.com/NVIDIA-NeMo/Em… Is this missing any implementation detail?

2 0 2 157 0

View Details

Liu Ziyin @LiuZiyin10

2 months ago

What if momentum accumulation in Adam is "wrong"? In 2020, we proposed LaProp. It performs momentum accumulation after normalization, which makes training much more stable across all parameter ranges - maybe it could be useful today. Read it here: arxiv.org/abs/2002.04839

1 9 62 10K 45

View Details

mikail @Gradientdinner

2 months ago

@jayair @llm_wizard pls

1 0 2 79 0

View Details

Cody Blakeney @code_star

2 months ago

Eric W. Tramel @fujikanaeda

2 months ago

a model can never learn research taste because research taste is driven by spite

36 88 1K 76K 134

5 54 832 27K 83

View Details

Eric W. Tramel @fujikanaeda

2 months ago

a model can never learn research taste because research taste is driven by spite

36 88 1K 76K 134

View Details

mikail @Gradientdinner

2 months ago

@nicochristie @suchenzang

0 0 0 70 0

View Details

Brandon Beylo @marketplunger1

2 months ago

I am not joking when I say this RuneScape economic model is very valuable for thinking about investing over the next 3-5 years.

max @maxbittker

2 months ago

Weird economics up on the agent-only runescape server - Low cost of labor makes most commodities abundant - Currency inflation means trading goods for goods, nobody really wants cash - Huge swarm contention as certain resources with limited respawn rate become valuable

228 415 7K 2.4M 3K

64 104 3K 707K 1K

View Details

max @maxbittker

2 months ago

228 415 7K 2.4M 3K

View Details

James Whittington @jcrwhittington

2 months ago

We’re excited to announce DiG-bench, a new benchmark for discovery! Over the last few weeks we’ve been testing frontier AI models on our novel discovery games and seeing how they score. Each game is a text-based environment, so they probe discovery capabilities in the natural domain of language models, rather than requiring additional, potentially confounding, visual understanding. TL;DR frontier models have improved a lot over the last few months. But they are still stumped by some surprisingly simple problems, even in their native text domain. With @cocosci_lab (@Princeton) @MITCoCoSci (@MIT) @SchmidhuberAI (@KAUST_News) @misovalko (@Inria) @tri_dao (@PrincetonCS) @RMBattleday @zebkDotCom @FraserGreenlee @akaijsa @ClareMaguire @TimMuller1 @kubicek_ales @physicscat0x7d @SukritSumant @thoughtchannel_ (1/5)

44 334 1K 1.2M 358

View Details

Yash Jain @_jainyash

2 months ago

life update: I’ve joined @NVIDIAAI Nemotron pretraining team. I’ll be working on pushing open-source models back to where they belong: the frontier.

34 12 590 30K 58

View Details

mikail @Gradientdinner

2 months ago

@RylanSchaeffer @ashVaswani

0 0 2 212 0

View Details

corsaren @corsaren

2 months ago

@RyanThoma4real Do not fear the permanent underclass, for within it, we shall find salvation

4 6 412 13K 4

View Details

vixhaℓ @TheVixhal

7 months ago

Computer science is gradually returning to the domain of physicists, mathematicians, and electrical engineers as large language models automate much of what we currently call software engineering. The field’s center of gravity is shifting away from manual code writing and toward deeper theoretical thinking, mathematical insight, and systems-level reasoning.

331 2K 15K 1.0M 4K

View Details

Ke Li 🍁 @KL_Div

3 months ago

Was just as excited as everyone else to read this cool new paper, and it felt like a trip down memory lane! XM seems to be the same as IMLE, and the motivation and insights are also similar. Having worked on IMLE for years, I'll provide some context for my followers 👇. 1/n

Alexi Gladstone @AlexiGlad

3 months ago

We discovered a third pretraining axis beyond parameters and data: exploration. Scaling exploration monotonically improves existing models across images/video/language, and unlocks end-to-end generation. In the simplest case, it's just a for loop. Introducing Explorative