Rachel @rayefull

Sometimes people who work in the extended AI ecosystem will say to me "I am not focused on RSI and xrisk because there is nothing I can actually do to stop it. I am going to trust the people in the lab know what they are doing and have it under control" but when I talk to friends in the labs they are like "I am buying land under a different name a few hours outside the city so when the public learns what we have done and revolts against lab employees I will have somewhere safe to go" or "I am applying to nz/aus citizenship so I have another passport if things get bad" or "I am no longer excited for each new model release because it tips us closer and closer to loss of control"

Sometimes people who work in AI ecosystem will say to me 'yeah my pdoom is low because idk if AI will literally Terminator us all. However I am very worried that 1billion people will die in an AI assisted disaster such as a nuclear cyberhack' and im like omfg guys

something funny about how legally it is vastly harder to build a duplex in San Francisco than it is to build a machine superintelligence

Zac Hill @zdch

a day ago

This year, four companies will spend roughly $730 billion building AI. The only law aimed squarely at what they’re building is a California statute asking them to post their safety plans online.🧵

5 2 27 8K 1

There’s a missing mood with ai safety and comms where I’m feeling like why aren’t people trying to win? Everything feels so defensive and skirmish and stuff when AI labs should not be race ahead is just the correct position.

I like places that feel like the edge of the earth. Australia and San Francisco both have this property

Big news. My take: Anthropic deserves some credit for actually proposing a pretty reasonable AI safety bill; that's more than other frontier labs have done. But they don't deserve *too* much credit, IMO; this bill is weaker than the Trahan-Obernolte FRONTIER Act in some key ways, and since the political momentum has shifted significantly in the direction of safety since FRONTIER was introduced back in July, I think new bills introduced today should ideally go further than FRONTIER was able to go rather than regressing. To be fair to Anthropic, the main places where this bill is significantly weaker than FRONTIER are (a) rulemaking (very strong in FRONTIER, almost nonexistent in Anthropic's bill) and (b) the emergency shutdown authority. Even if you take safety seriously, it might make some sense to dial those authorities back if you didn't trust the executive branch to wield them responsibly, because those are the easiest provisions for a hypothetical President who hypothetically might not like a certain frontier AI company to abuse. There are some improvements over FRONTIER as well; for instance, this bill does specify that auditors are employed on a continuing basis and has better provisions regarding auditor access to information. Happy to see that from Anthropic (although I think their access provision is still a little undercooked and has Fourth Amendment problems). Overall, I think the embedded auditor bills that get introduced in the coming months are going to be significantly better and more ambitious than this, and I hope Anthropic supports those bills instead of opposing anything that treats catastrophic risks like the serious national security issue they are. Still, it's encouraging to see a frontier lab come this far.

POLITICO @politico

19 hours ago

Anthropic quietly pitches its own AI safety bill to lawmakers politi.co/4zxDFhl

15 50 105 172K 83

Charlie's analysis is right but he is nicer than I am - I found myself viscerally angry at multiple points reading this bill. There are some good and sensible parts, but overall this legislation is not remotely close to meeting the moment and I overall think passing it would make us less protected against catastrophic risks than we are under existing law. Two parts that stand out as particularly infuriating when taken together: 1. Some of the preemption provisions are remarkably far reaching and consequential, and in addition to preempting legislation like SB 53 would also remove general powers that state courts have to enjoin defendants and protect the public. Most outrageously, it has a provision that looks like its intended to stops states from enforcing laws or judicial decrees that "suspend the development, deployment, or internal use of an AI model" to protect their citizens from imminent catastrophic harms. Such powers to block and suspend model deployments would then only reside with the federal government. 2. The federal governments powers to restrict the development, deployment, or internal use of a frontier model has to meet an absurdly high bar. First, there needs to be an imminent risk of a catastrophic harm (meaning killing more than 50 people or causing more than a billion dollars in damage). But thats not enough! The agency head needs to make a detailed determination with no deference given to show why no alternative would prevent the imminent catastrophe. Next the court is also explicitly told to consider the harms to the developer and other harms from restricting the model, but no considerations in the other direction. This is absurd! For all of its warning about the potential for AI to cause a catastrophe that kills literally everyone on the planet, Anthropic's proposed federal law likely makes it harder than under existing law for a court to get an order blocking or restricting the development of frontier models? WTF?? There are also a bunch of others things that I would like to see improved - the substantive details of the auditing section and the potential for rule-making on substantive safety standards both come to mind - but I think these stand out as particularly indefensible.

Charlie Bullock @CharlieBull0ck

18 hours ago

Big news. My take: Anthropic deserves some credit for actually proposing a pretty reasonable AI safety bill; that's more than other frontier labs have done. But they don't deserve *too* much credit, IMO; this bill is weaker than the Trahan-Obernolte FRONTIER Act in some key

14 37 224 47K 107

@ChanaMessinger makes sense

You can tell from the QTs how desperate people are for any crumb of hopium that lets them keep doing what they're already doing (racing to RSI at full throttle) with a clean conscience. "The faster we race, the sooner we solve alignment!"

Nick @nickcammarata

2 days ago

interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came

825 3K 19K 4.0M 20K

@grnsleeves idk set up an ai to clear the spam out of ur inbox, we live in beautiful times

I really wish more people put their email on their website instead of just social media links, cause I in fact sometimes want to get in touch with people and then have to lift it from their nested resume

@hamandcheese Tracks. I met up with my mom and sister on the beach (who are in white linen pants) and the first thing they say to me is ‘you look like you’ve escaped a convent’

I feel like I look like a gothic Victorian lady who was prescribed fresh air out on a beach day

FAI's @hamandcheese argues that OpenAI and Anthropic should be allowed to coordinate in ways normally prohibited by antitrust law to prepare for superintelligence: "When you’ve crossed into these thresholds of recursive self-improvement, there’s no natural checkpoint, no natural milestone where you can pull back and take a look at the model, evaluate whether you can still control it." "Real coordination is going to be needed above and beyond just, do we have auditors in our company? And real antitrust protections are probably the only thing that will enable it. On the populist right, they hear antitrust protections for big tech and they think Section 230. This is not what I have in mind." "Under the Defense Production Act, 708 agreements are industry consortia. We just finalized one with the Department of Energy for the nuclear industry, where they can now coordinate in ways that would normally be in violation of antitrust, under very rigorous oversight by the FTC and DOJ." "In my dream scenario, bring Anthropic, OpenAI together to pool their capital to stand up a third-party organization, let independent evaluators use that as their launchpad. Some kinds of standards for when do we have a green light versus a red light. That has to be done transparently, in the open, and not purely at the discretion of an unnamed bureaucrat." @JoinFAI

@albrgr oh no

@hisaflog100 Why?

The internet is so slow where I am in Australia. It is under 4 Mbps. Everyone is chilling but I am going to crash out. I am heading into town to buy more internet from Telstra. It is cool I can just buy a puck of internet and put it in my pocket and solve my problems.

@JazzRaytion It’s less a network issue and more a ‘I am visiting my grandparents and they’re not participants in the global economy in the same way’ kinda thing

The Information is reporting that after the HF attack, OpenAI shifted about 10% of its total compute to safety. I think it would be very good for OpenAI and other companies to release much more information publicly about how they are allocating their compute and how they define different categories - compute allocation (to inference vs safety/security vs internal R&D etc) is one of the most important decisions companies make, and it would be great to see a race to the top towards spending more compute on safety and inference, and less on automating capabilities R&D.

@Actuallykeltan it’s truly a lovely Time Machine