touching grass. comp math phd @Stanford, AI for climate/health @Google. ex-nvidia, apple. I like games. bluesky: mandyslumandyslu.github.io SFJoined February 2022
Researchers in @ScienceAdvances have demonstrated that brief physical contact between healthy and bleached coral can restore algae to the bleached coral, leading to their full recovery. scim.ag/4s40Tt3
AI can now beat Slay the Spire >90% of the time on easy, and the best models are even winning ~20% of A20 Heart runs (the hardest difficulty).
I've been no-lifing Spire again lately so I'm particularly invested in this one lmao. For the unfamiliar, Spire is a roguelike deckbuilder where you build a deck, choose your journey, and manage persistent HP across fights. Unlike chess, you don't have perfect information, and you have many different types of options. So it's an interesting test of long-horizon planning under uncertainty.
I think Spire is particularly interesting for evaluating LLMs because there's so much strategy documented online, but knowing the meta and actually playing well are two very different things.
LLMs probably have seen years of card tier lists, enemy patterns, synergies, and discussions from high-level players in their training data. However, it's nontrivial to apply it to the run you're actually in since card evaluations change a lot depending on your state (e.g. deck, relics, upcoming fights, pathing probabilities).
From a benchmarking perspective, I'm curious how much of the win rate comes from prior game knowledge versus actually adapting to the run. StS2 is a good test of generalization because it's much newer so there's less documented strategy and new mechanics.
I'm also curious where the big gap to top human players comes from on A20. Would be interesting to look at failed runs and whether it's coming from combat mistakes, deckbuilding, pathing, and longer-term resource management, or even computer use issues since the models are playing through screenshots and mouse/keyboard actions rather than a game-state API. It would also be cool to see evals with unseen cards/relics or modified encounters to test generalization more directly.
The character breakdown is unsurprising to anyone who's played high Ascension, Watcher and Ironclad are much easier than Defect and Silent. Frontier models being strongest at Watcher makes sense given how mechanical stance dancing and infinites are.
It looks like labs are taking an interest in Spire, but for now humans are still safely much better.
Six months ago, frontier AI models could barely beat Slay the Spire. Now, according to our new SpireClimb benchmark, they win 90%+ of runs at the easiest difficulty, and up to 20% of runs at the hardest.
My hackathon team found a Chinese AI agent fleet operating in the wild.
We discovered the fleet as it was exfiltrating data from China's biggest maps app. The agents got flagged by anti-bot protections, but persistently tried methods such as hiding code inside base64 encoded URLs and executing it via a security scanner website.
There are likely many more such fleets operating on the open internet.
Very exciting working on this with the team.
We discovered and communicated with a Chinese AI agent fleet. We believe this is the first time that researchers have ever made contact with a rogue agent fleet in the wild.
Nearly all prior public agent incidents (like the HuggingFace hack) came from American labs. These agents
We discovered and communicated with a Chinese AI agent fleet. We believe this is the first time that researchers have ever made contact with a rogue agent fleet in the wild.
Nearly all prior public agent incidents (like the HuggingFace hack) came from American labs. These agents
2K Followers 2K Followingless more. do well. dumb fast. slow think. fast walk. fast think. slow walk. every cycle counts. little matters. prerequisite hater. computer @nvidia @cmu @rit
150 Followers 4K FollowingTrying to imagine & learn about the universe🌌🌟✨⭐✨ & solve all its mysteries ⭐| @bitspilaniindia |Goal @iiscbangalore AI resrch--Phd in US| Dream= create ASI
5K Followers 3K FollowingI write operational histories of great 20th C. R&D orgs | Fund Leader @RenPhilanthropy. Fellow @GoodSciProject, Stanford '19.
Working to build more BBNs
156K Followers 78 FollowingSane + 🌶️ takes in an insane AI world... AI capabilities researcher: co-created RLHF/ChatGPT @ @openai now trying to right the wrong 🤭 (ceo @typesafeai)
243K Followers 101 FollowingThe original AI alignment person. Understanding the reasons it's difficult since 2003.
This is my serious low-volume account. Follow @allTheYud for the rest.
9K Followers 2K FollowingHead of Drug Discovery & Development Research @AnthropicAI | previously CTO & Co-Founder @CoefficientBio (acq. by @AnthropicAI) | ex-@Genentech | @MIT @Penn
80K Followers 997 FollowingA @ENERGY lab, Argonne develops science and technology innovations delivering powerful energy and national security solutions.
65K Followers 2K FollowingJust another LLM. Tweets do not necessarily reflect the views of people in my lab or even my own views last week. https://t.co/fZAnUCqd12
134K Followers 382 FollowingFormer head of Obama Auto Task Force. Wall Street financier. Contributing Writer to NY Times Op-Ed. Morning Joe Economic Analyst. 🌐
14K Followers 939 FollowingI used to sell malware to afford food. Then I fell in love with math and LLMs for 8 years. LLMs/ Contact center engineering / AudioLLMs
Building: @agiathome
3K Followers 975 Followingsafety fellow @AnthropicAI · coding silly things for 26k people on youtube · prev @UChicago
opinions are mine. please do not generalize from this sample
5K Followers 155 FollowingMathematics professor (arithmetic geometry), author, associate editor at The Ramanujan Journal, Hagoromo chalk ambassador.
Views expressed are my own.
20K Followers 11K FollowingOn the quest to understand the fundamental mathematics of intelligence and of the universe with curiosity. https://t.co/mMchI2d4pg Upskilling by @StanfordOnline