I spent last weekend at the Takeoff Conference in DC, trying to understand what’s currently going on with AI policy.
The ideas I heard can all be grouped into one of two high-level buckets:
Proactive proposals. Proposals in this bucket aim to regulate AI before things get crazy. However, the ones I heard all rely on traditionally slow-moving institutions (e.g., the courts, the states, or Congress). As a result, they share an implicit assumption: that timelines are long enough (either by default or as a result of an intentional slowdown) that we can afford to wait at least another year or two for policies to take effect.
Reactive proposals. Proposals in this bucket focus on making sure we’re ready to respond to emergency situations after AI gets crazy, which makes them compatible with much shorter timelines than the proposals in the first bucket. However, these proposals all rely on a different assumption: that we’ll have the chance to respond to another Hugging Face-style warning shot before it’s too late.
Importantly, I didn’t hear a single proposal aimed at the world where timelines are short and, by the time we get a clear warning shot, it’s already too late to act (like the world Paul Christiano describes in his interview with Dwarkesh). My read here is that some of the people I spoke with just don’t take short timelines seriously. Others do believe we’re in a short-timelines world, but doubt we have time to implement meaningful proactive policies before a big emergency (because the White House seems unlikely to act and Congress won’t move fast enough). So even these folks have resigned themselves to a more reactive posture and are hoping the labs decide to slow themselves down.
My goal with this post is to summarize the different policy approaches I heard about, and to sketch out my basic thoughts on this “short-timelines-no-warning-shot” gap.
Proactive proposals
Civil suits
Civil suits offer a path to “private lawmaking” that I hadn’t considered before this conference, but that I now feel quite bullish about.
The basic idea is that, if a suit is brought by the right plaintiff, its settlement can impose binding, court-enforced rules on how a lab operates, thereby creating de facto governance and regulation without requiring any new legislation.
There’s strong precedent here. In 2019, the FTC sued Facebook for violating its own consumer privacy promises. It used the settlement to not only fine Facebook $5 billion, but also to restructure Facebook’s governance: it created an independent privacy committee on Facebook’s board and started requiring outside audits, among other things.
Of course, not all suits are capable of forcing this sort of change. But there are already a couple of promising paths (one live and one potentially in the works):
California v. OpenAI: earlier this week, California’s attorney general sued OpenAI over the Hugging Face hack. The suit is quite broad: it doesn’t specify exactly what practices it would like to see OpenAI change. But, it does ask the court to order OpenAI to abate the risk of escaped agents. This seems like a productive opening for private lawmaking focused on safety, testing, and control.
The FTC investigation: the FTC is investigating safety risks at several frontier labs, including OpenAI and Anthropic. I’ve heard a few folks argue that, if it wanted to, the FTC could turn the “morally binding” SI accord into a legally binding set of rules, since failing to follow through on such visible public commitments could count as deceptive advertising.
One person I spoke with argued that civil suits have a better chance of producing real outcomes if the labs feel that regulation is coming for them one way or another: if the labs see meaningful progress being made at either the state or federal level, they may actually welcome the opportunity to settle (since it gives them a greater say in their own regulation).
State legislation
I didn’t deep-dive any specific state-level bills slated for the rest of this year or 2027, but I was surprised by the level of general optimism around state legislation. It seems like the states have a major role to play in the short-term regulatory picture.
As far as I can tell, the optimism here is the result of two things:
Precedent: recent history shows that state-level AI laws (e.g., California’s SB 53, New York’s RAISE Act, and now Illinois’s AI Safety Measures Act) can create nationwide standards. This is because, rather than maintaining separate standards for each state, the AI labs have been taking the most stringent state laws and applying them nationwide.
Speed: state legislatures move faster than Congress.
Importantly, the case for state-level legislation seems particularly strong for regulation concerning a lab’s internal practices. For example, if you want to pace recursive self-improvement (RSI) by, say, mandating a minimum % of compute that must be allocated to paid inference, or by regulating the use of non-externally-deployed models for internal AI research, you probably only need to pass a law in California, since that’s where almost all frontier research happens. And sure, labs could just move their research operations, but that would be a huge distraction and likely lead to a significant slowdown. So… mission accomplished?
Surprisingly, no one I spoke with seemed particularly concerned about state laws’ vulnerability to federal preemption or legal challenges. My read is that that’s because the next year or two matter an incredible amount, and the priority is getting SOME regulations in place even if they’re overridden later.
(Note: in September, Newsom asked for input on a few specific AI-related topics, including embedded evaluators, kill switches, and loss-of-control reporting. These items seem like reasonable starting points for any 2027 CA legislation.)
Federal legislation
At the federal level, the only AI-related bills with a real near-term shot seem to be export controls. Almost everyone I spoke with expects Congress to pass them soon, likely as part of this year’s defense bill (the FY2027 NDAA). There are 3 individual bills on the table, all of which, in one way or another, make it harder for China to acquire advanced AI chips / chipmaking technology (see Appendix if you want more detail on the individual bills).
I didn’t hear strong criticism of any of these bills, although that may be sampling bias. The general idea seems robustly good: every single person I asked thinks the US currently has at least a one-year lead over China, and export controls seem like a reasonable way to maintain or widen that lead.
It’s unclear which of these bills will make it into the final NDAA, but the recent Jensen and Lutnick scandals have given them new momentum. As for timing, the annual NDAA typically gets signed in late December: the export controls that end up passing should go into effect sometime between then and mid-2027.
(Note: as I was writing this, Commerce issued an export-control-related rule closing China’s remote-access loophole.)
Reactive proposals
Executive orders
Executive orders could be used for both proactive and reactive policymaking. The reason I’ve slotted them here is that, absent a major catastrophe, the current admin doesn’t seem willing to enact safety regulations via executive order.
Instead, a few of the people I spoke with are working to present Trump with solid, preplanned executive orders that he can quickly sign if shit does hit the fan. The idea makes sense: if there is a sufficiently bad AI-related catastrophe, Trump will want to take action – it’s better to do the work now to ensure the action he takes is productive (rather than letting the government figure it out itself or trying to muddle through a knee-jerk plan in the fog of war).
Interestingly, multiple people pointed to the Mythos export control debacle as a reason to be optimistic about this approach. It’s not that they approved of the actual actions taken. It’s more that the episode shows that the White House takes AI-related risks seriously and can move decisively when faced with (what it views as) compelling evidence.
You could imagine a world where, if the White House had had better policies at hand right when Mythos spooked them, they could have channeled their desire to act into something more productive. That’s the idea moving forward, just with higher stakes.
(Note: I didn’t speak to anyone who was willing to lay out the concrete details of one of these draft executive orders. The people who seemed to be working on them indicated that the specific details needed to be kept confidential. I’m not sure if that’s true or if I just didn’t meet the bar for disclosure, but either way, I would be quite keen to speak with anyone working on this who is willing to share specific details.)
Strategic reserves
There’s another complementary program that involves building up a reserve of technical talent pre-committed to working for the government if we find ourselves in an emergency situation.
Here’s my basic understanding. The government has a dearth of technical talent… this is already a problem, but it will be an even bigger issue when we’re trying to move quickly against advanced AIs. Without an emergency, it’s hard to sway lab employees to walk away from their current jobs and insane salaries to instead do thankless government work. But a lot of them are willing to pre-commit to making that switch if shit hits the fan. Basically, we’re building the Army Reserves for nerds.
I don’t quite understand the tactical implementation. The Intergovernmental Personnel Act Mobility Program (IPA) lets think tanks and other well-funded government-adjacent institutions hire folks and then place them in government positions while still covering their salaries. The only problem with this is that people need to be employed by the sponsoring organization for at least 90 days before they can IPA. That’s too slow for a crisis response.
It seems like a fast version of this would be to start signing reservists up as federal employees now, but just keep them on-call (which also means the government doesn’t have to pay them). FEMA already does something like this, so it’s not totally crazy. This would allow these researchers to be pre-cleared and to have negotiated conflict-of-interest waivers in advance so that they can start working right away.
One criticism here is that these pre-commitments aren’t typically binding. But as far as I’m concerned that’s not a downside; it just weakens the upside. Overall, this seems robustly good, and I’m strongly in favor of it.
Banning RSI
I spoke with someone about a plan to get Congress to quickly ban RSI via a stripped-down, weak-sauce bill that basically just says “it’s illegal to engage in RSI-related practices” without going into much detail about what those practices are or what the oversight and enforcement mechanisms should be.
This weakness is intentional: Democrats have already shown interest in banning RSI, and there’s a better chance of getting Republican lawmakers on board if the bill avoids any strong oversight or regulation.
While such a bare-bones bill might not do much to proactively stop RSI, it would give the DOJ a bigger hammer with which to react: this bill would give the DOJ clear authority to intervene (e.g., halt training) and dole out punishments if it ever did suspect RSI.
I think it’s an interesting idea, but I don’t actually expect it to work. Even if you could get it through Congress, it feels like the exact type of thing the White House would veto, citing concerns about race dynamics with China. And those concerns are potentially valid!
Other stuff
One of my high-level takeaways from the conference is that our option set opens up a lot if something shifts us to wartime footing. State governors have emergency authorities, and even Congress can move quite quickly in the face of an emergency. Obviously, COVID should not be held up as a best-in-class response to an unfolding crisis, but if you’re just looking at the speed at which Congress was able to pass legislation, they moved much faster than they typically do. Or look at the 2008 financial crisis. When Congress needs to move fast, it can.
Also, maybe this is naive, but you could imagine a world where the evidence of AI misalignment / takeover is so conclusive that labs are willing to coordinate a pause in spite of antitrust risk (more on this below).
The “short-timelines-no-warning-shot” gap
I’m sure this list is incomplete, but it represents my current understanding of the policy landscape: we have a handful of slow, proactive policy proposals and a handful of fast, reactive policy proposals. We don’t have a clear plan for a world with short timelines and no warning shots.
So, at the risk of ending this whole thing with the most ice-cold take possible, it seems like the most important thing to do right now is to slow the pace of progress and buy ourselves some time, so that the proactive proposals have a chance to take effect.
The most common objection I hear to this is: what about China? I’m sympathetic to that concern, but as I mentioned above, every single person I spoke with this past weekend thinks the US currently has at least a one-year lead. That’s not a huge lead, but it does seem like we have at least a bit of room to pause or slow down without handing the race to China.
It’s also not clear how much our current one-year lead is actually worth. The consensus seems to be that frontier labs have such bad security that China will just steal their weights / research whenever it decides it’s time. So continuing to race might not widen our lead. Instead, it might actually shrink it. It certainly feels worth burning a little bit of our lead to shore up security.
Unfortunately, given that the White House is unlikely to act and Congress moves at a glacial pace, it seems like any near-term slowdown will have to be voluntary. The labs say they’re willing but have so far failed to coordinate. They can’t even agree on whether or not antitrust is the main barrier (Anthropic has asked for a waiver, while OpenAI says one isn’t needed).
So if I had to choose one thing to take action on in the near term, it would be calling the labs’ bluffs – solving the (supposed) antitrust issue and giving them a legal way to coordinate a slowdown. To be clear, I don’t think solving the antitrust issue guarantees productive inter-lab coordination. But even if it doesn’t, that’s useful information: a lab that claims to want to coordinate in theory but then refuses to do so in practice does not actually want to coordinate. So irrespective of the end result, this feels like a valuable problem to work on.
And even though I don’t yet have a clear sense of the right solution here, there’s just no way the antitrust problem is unsolvable (Matt Levine has a characteristically funny take on this and AIUC has done some great research on at least one possible solution). Given its importance, this is a topic I hope to write about soon.
Appendix: export controls
Goal: Make it easier to detect smuggled chips
Summary: Requires location verification on export-controlled chips; adds mandatory reporting for chipmakers if they discover chips in unexpected locations (i.e., smuggled)
Goal: Stop China from buying advanced chipmaking equipment (e.g., for DUV immersion lithography) from countries like Japan and the Netherlands
Summary: Gives allies 150 days to voluntarily match US export controls on selling (and servicing) advanced chipmaking equipment to China. If they don’t, the US will apply export controls directly to any foreign-made advanced chipmaking equipment that uses US tech or components (using the authority of the existing “Foreign Direct Product Rule”)
Goal: Let Congress block the sale of advanced AI chips to China (and other adversaries)
Summary: Revokes any existing licenses to sell advanced chips to China (e.g., H200s), bans the sale of Blackwells to China for at least 2 years, and routes future licensing through a 30-day Congressional review (basically treats the sale of advanced AI chips like we currently treat the sale of weapons)

One useful note on state pre-emption, the doctrine that would most likely have lead to successful legal challenges (dormant commerce clause), got largely defanged by a case called National Pork Producers in 2023.
Lots of good ideas here… a bit surprising that so few people are thinking concretely about how to succeed as we move into an emergency situation. If SHTF there more likely than not will be a pretty much unlimited activation of state capacity, which can probably be an OOM more or less effective depending how competently it’s wielded.