Great work by friend @RunhanH ! It is exciting to see how frontier LLMs change the status of long-standing problems in the field, which in this case is the way to explore during "learning while deploying". Code-as-Policies is a start, and a major component of robotics tool-use.
Robotics has spent years scaling demonstration data and model size. What happens when we scale EXPLORATION for robot agents?
We're incredibly excited to share RAX, an agentic robotics system that learns by scaling online exploration.
A coding agent connects directly to the
Wrist cameras give robots close-up views, but only where the hand happens to be.
What if the policy could decide where to look?
Introducing FoveaVLA: a VLA that replaces the wrist camera with learned foveation actions.
FoveaVLA predicts where it will need to look next, then crops that region from the full-resolution external image. The full view gives context; the foveal crop gives close-up detail.
With one external camera and no wrist camera, FoveaVLA reaches 70.8% success on 48 real-world trials, vs. 52.1% for π0.5 with a wrist camera.
• No additional demonstrations
• No camera calibration
• No wrist camera
Paper, code, and models are all open 🧵
@Yuchenj_UW Another important, if not the most important thing is they had good product managers. The success of muse is because it is a very appealing product with reasonable model capabilities.
(1/5) Everyone says dexterous manipulation needs real-world data. These policies were trained with none. Our robots mesh gears, thread nuts, and insert pegs zero-shot from RGB, and the same recipe works for locomotion across terrains much harder than before.
The missing piece
I generally find that nobody really cares about papers, so I’m trying to add something a bit more human to my personal website.
If you think my research is cooked, at least enjoy some things I actually cooked :)
joyemang33.github.io/cooking/
🙏 OpenWAM is a large undertaking, made possible by the dedication and collective effort of the entire team. Thank you to everyone who contributed to this project, couldn't have done it without y'all!
🤞 I’m grateful for the opportunity to have co-lead this project with @YuranWang_rise. Special thanks to the amazing adviors: @zhaohang0124, @linshaonju, @HaoDong123, and @LotusSapphire for their advice and guidance throughout the project.
🧵14/14
📦 We release OpenWAM as a full stack empowering WAM research:
- Modular infrastructure for training, inference, deploy, and eval.
- OpenWAM-α pretrained and post-trained weights.
- Study checkpoints for exploring the design choices.
- Evaluation protocols and data recipes.
🥞 We hope researchers can use the stack to reproduce the results, adapt OpenWAM-α to new tasks and embodiments, and test new ideas for representations, world–action interaction, and embodied pretraining.
🔭 There is much more to explore. We hope OpenWAM-α serves as a strong, reproducible baseline and OpenWAM-Infra makes that next round of experiments easier to build.
🌐 Project: openwam-official.github.io
📄 Paper: arxiv.org/abs/2609.07398
💻 Code: github.com/OpenWAM-Offici…
🤗 Models & data: huggingface.co/OpenWAM
🧵13/14
🤖 World Action Models predict the future to act better. However, WAM research remains fragmented in code, ad hoc in design, and unprincipled at scale.
💡 Today, we release 𝗢𝗽𝗲𝗻𝗪𝗔𝗠: 𝗔𝗻 𝗢𝗽𝗲𝗻, 𝗠𝗼𝗱𝘂𝗹𝗮𝗿 𝗘𝘅𝗽𝗹𝗼𝗿𝗮𝘁𝗶𝗼𝗻 𝗧𝗼𝘄𝗮𝗿𝗱𝘀 𝗦𝘆𝘀𝘁𝗲𝗺𝗮𝘁𝗶𝗰 𝗪𝗼𝗿𝗹𝗱–𝗔𝗰𝘁𝗶𝗼𝗻 𝗠𝗼𝗱𝗲𝗹 𝗣𝗿𝗲𝘁𝗿𝗮𝗶𝗻𝗶𝗻𝗴 to change this.
🌐 openwam-official.github.io
🧵1/14
426 Followers 2K FollowingStill believing that the right person will find me.
04男大 短期主义
Curious mind. Clear thinking. Always learning. No politics, no hate, just ideas and growth.
803 Followers 7K FollowingMachine learning engineer working on signal processing, NLP and (some) computer vision.
Trying to make the most of my measley 3090 and 4 Sparks.
2K Followers 356 FollowingRobotics Research @physical_int | EECS Rising Star; Prev CS PhD @UW built my 🥢 robot; intern at MetaAI, Microsoft Research, Google Search
16K Followers 5K FollowingFull of childlike wonder. Teaching robots manners. RL Lead @ Apptronik. UT Austin PhD candidate. Past: Boston Dynamics AI Institute, NASA JPL, MIT ‘20.
19K Followers 3K FollowingMachine learning & AI podcaster, community builder and all around enthusiast. Creator of the @TWIMLAI Podcast, TWIMLcon, TWIMLfest & the TWIML Solutions Guide.
1K Followers 442 FollowingPhD student @imperialcollege
Building : https://t.co/omsy6QXYvy (20k🌟)
Bilibili: 卡普迪姆 · 30k+ follower
Be a role model, not an idol.
1.9M Followers 2K FollowingProfessor of Neurobiology and Ophthalmology at Stanford Medicine • Host of Huberman Lab • Focused on science and health research and public education
1K Followers 304 FollowingMTS @WaldenRobotics. Previously @StanfordAILab PhD, @WeAreHRT, @ToyotaResearch, @berkeley_ai. I can teach your robot new tricks.