Tentative thesis: China is unlikely to accept a subordinate position in AI capabilities to the US, just as the US is unlikely to accept a subordinate position in AI capabilities to China or anyone else. 

This thesis suggests that there are two likely paths for the future international AI (non)regulation:

1) a continuing race between the US and China, perhaps joined by some late entrants, possibly with some rules established (e.g. mutual ban on autonomous weapons or something like that), or

2) a “pause or stop deal” that will establish some sort of ceiling on capabilities, which will be equal for China and the US.

What I think is far less likely is a deal in which China would accept lower AI capabilities than the US.

I simply don’t see a good reason why Chinese leaders would accept subordinate position. They know that China has the ability to catch-up to the US in various technological domains, as evidenced by the fact that in, like, 1990, China was behind in more or less everything, and now they are pushing the technological frontier in many areas. 

Some estimates floating around the web suggest that, if the US would stop AI development now, China could reach current US capabilities within months. That is maybe overly optimistic/pessimistic depending on where you stand, but I very much doubt that catch-up would take more than a decade. And Chinese government, being patriotic about the abilities of the Chinese nation, probably will not have an absurdly pessimistic estimate.

Moreover, the Chinese regime is in many ways oriented around this idea of catching up (this is a big difference between China and EU). It is a central plank of the official historical narrative of the People’s Republic of China (see for example preamble to their constitution: https://english.www.gov.cn/archive/lawsregulations/201911/20/content_WS5ed8856ec6d0b3f0e9499913.html) that the century between First Opium War and the Communist victory in the Chinese civil war in 1949 was the century of humiliation, during which various imperialist powers exploited China due to the fact that it was technologically backward country unable to stand up to them. I happen to think this narrative is largely correct, but even if it isn’t, what matters is that Chinese government believes it and is determined to never again allow China to be put in the position where a technologically superior power could walk over it.

This has some implications:

A) Since China is currently behind, any sort of “pause or stop deal” between the US and China would likely have to involve either the US stopping further development and allowing China to catch up, or the US somehow crippling its own capabilities to a current Chinese level. Of course, for either option, the US would have to place a lot of trust in the Chinese government.

 B) As long as China is behind, export controls on hardware and other attempts to limit Chinese capabilities are at best of doubtful utility as a way to get to a "pause or stop" agreement. In fact, imho there are making a deal less likely. They antagonize Chinese leaders, make them even more paranoid and less inclined to a constructive agreement. But also, to the extent they are successful in limiting Chinese capabilities, they make it less likely to that the US government would agree to an equal deal, since that would mean sacrificing larger lead, which is harder to politically accept for the US. Perhaps this is a reason why nuclear arms treaties in the Cold war were agreed only after the USSR more or less caught up to US nuclear arsenal?

Now, attempts to limit Chinese capabilities can still be seen as useful if you think it is important for the US to “win” the competition, for example because there are reasons to believe US AI will be safer. Or because you think the world would be a better place if the US “wins”, even though both countries would develop similarly dangerous systems. Or of course if you think that the US victory would be preferable even if American AI would be more dangerous than Chinese AI.

Btw. for a well written snapshot of Chinese opinion/vibes, I recommend this: https://www.chinatalk.media/p/how-anthropic-became-chinas-goliath

I should mention there is an argument that many mental health interventions are in fact harmful, mainly via a priming mechanism where when people start to consciously preoccupy themselves with their own mental health, that itself causes psychological suffering. It is a sort of Buddhist argument, I guess.

Abigail Shrier's Bad Therapy (2024) has probably most discussed version of that case, though I have not read the book, and I am extremely epistemically uncertain about all this, in a way that makes me reluctant to either support or oppose interventions in this area.

I am late to this but anyway I'd like to point out that main and huge dowside of export controls is imho that they are poisoning the atmosphere for international cooperation on AI (also on other things), for reasons I tried to explain here: alesziegler's Quick takes — EA Forum

"But I think maybe the cruxiest bits are (a) I think export controls seem great in Plan B/C worlds, which seem much likelier than Plan A worlds, and (b) I think unilaterally easing export controls is unlikely to substantially affect the likelihood of Plan A happening (all else equal). It seems like you disagree with both, or at least with (b)?"

Yep, this is pretty close to my views. I do disagree with (b), since I am afraid that controls might poison the well for future Plan A negotiations. As for (a), I don’t get how controls help with Plan C, and I don’t think Plan  B/C worlds are much more likely than Plan A, as of now. But I do agree with you that controls help with Plan B, so if you see it as the main hope, I understand why you are supporting them.

(btw. how does one do a proper blockqoute here?)

Thx for your perspective; I should be upfront that my confidence in my own case is far from ironclad. Anyway.

From the ryan_greenblatt’s article you’ve linked, I think safety plan A, based on achieving international agreement, should be tried before going for plan B, based on achieving and using a secure US lead, while in “plans” C and D it doesn’t matter whether the leading company is American or Chinese, so slowing down Chinese development is useless for those “plans” (they are in fact more like scenarios than real plans).

I do agree that the US political system is in an important sense better than the Chinese political system, but my prior is that if superintelligence is developed before, let’s say, 2050 (and that is very optimistic), it is likely to go badly completely regardless of which country it’ll come from.

I take your point about slack being potentially useful. Theoretically, I can imagine the following sequence of events: a) the US AI industry crushes Chinese competition, then b) the US government, feeling secure in the US lead, imposes sensible safety regulation on companies. If it were smart about it, it would at the same time propose an international regulatory framework that the rest of the world would be prepared to sign on to, as an alternative to unrestrained US domination. In effect, this would be tantamount to getting so much leverage over China that they would drop out of the race, and then hoping that the US government would use its advantage to push for safety, instead of using it in some other way.

However, imho this is a plan that should be pursued as a first option only in circumstances where you are really confident it’ll work, since the consequences of trying and failing are likely to be dire, as per my original post. And I am not confident it will work.

A much better order of operations would be to 1) try to negotiate with China to establish an international regulatory framework (plan A), with export control and other stuff being imposed as something that is explicitly linked to China not agreeing to that framework, in the same way sanctions on Russia are imposed explicitly because its aggression against Ukraine, and 2) only if they refuse, try to crush them (plan B).

When political will in the US to try for plan A is lacking, I think waiting until circumstances make that plan realistic while preparing the groundwork for it is a better strategy than going straight ahead for plan B.

Hi, thx for engaging, but I guess we’ll have to agree to disagree. 

I do agree that chip controls would make a lot of sense as a source of leverage in negotiations over an AI pause in a world where China would be trying to build superintelligence and the US would be trying to force them to do a mutual pause. But we don’t live in that world.
The Biden administration had no intention to agree to a cooperative pause when it imposed chip controls, and the Trump administration has somehow even less than no intention to pause; Trump seems to be going all-in on AI development.

You might object that controls are good because they are sort of increasing the supply of leverage which in the future could be used to do a pause, and that is true, BUT to do a deal, both leverage and trust are needed. And imho even without controls, the US has a lot of leverage over China just by the sheer power of its global hegemony.

So, leverage over China is already relatively abundant, so to speak, but Chinese trust is what is relatively scarce (as is, of course, American trust in Chinese intentions). So, I am skeptical that steps which build more leverage over China but decrease Chinese trust in American intentions get these countries closer to cooperation.

And I disagree that lifting controls doesn’t recoup Chinese trust. It does not recoup it fully, as if controls never happened, but I do think that it builds trust relative to a situation of continuing controls.

To use a somewhat strained example with which most are surely familiar, when Trump put up his Liberation Day tariffs, prices of various financial assets fell off a cliff, but when he partially reversed the tariffs, prices partially recouped. How come, if Trump has shown that he is untrustworthy and might change his mind any minute? Well, many investors are evidently betting he won’t do that, and so far they’ve been correct.

Imho it is a fairly general principle of human relationships that for maintaining trust: never doing a trust-damaging action > rescinding a trust-damaging action > continuing a trust-damaging action.

Also, Chinese-American relations are themselves a vivid example that trust can be rebuilt: in the 50s, they were at rock bottom due to various trust-destroying actions, notably in the Korean War, where the Chinese army attacked UN forces advancing toward the Yalu River and pushed them back from what is now unfortunately North Korea. But, skipping ahead, by 2000, after a lot of trust building, the US granted China Permanent Normal Trade Relations. Things went downhill after that, but that could change again. 

I also disagree that chips used to train/run LLMs are in the same category as F-35s or nuclear material; imho they are more like steel or oil, commodities obviously important for military purposes but with extremely broad civilian applications, and actions like an oil embargo are considered to be hostile acts, usually only deployed against pariah states, unlike an “F-35 embargo.”

Finally, imho you misunderstood what point I was trying to make (ok, that’s on me, I should’ve been clearer) about pariah states – it is not that chip controls make China into a pariah state. They don’t. Rather, by deploying measures usually only used against pariah states, these controls blur differences between “normal” members of an international community and pariah states, thus devaluing the usefulness of not doing things that put the country on the naughty list, primarily for China but secondarily for everyone else.

Since a thing that puts countries on the naughty list is waging an aggressive war, I do think this development is bad.

Is the recent partial lifting of US chip export controls on China (see e.g. here: https://thezvi.substack.com/p/selling-h200s-to-china-is-unwise) good or bad for humanity? I’ve seen many takes from people whose judgment I respect arguing that it is very bad, but their arguments, imho, just don’t make sense. What am I missing?

For transparency, I am neither Chinese nor American, nor am I a paid agent of them. I am not at all confident in this take, but imho someone should make it.

I see two possible scenarios: A) you are not sure how close humanity is to developing superintelligence in the Yudkowskian sense. This is what I believe, and what many smart opponents of the Trump administration’s move to ease chip controls believe. Or B) you are pretty sure that humanity is not going to develop superintelligence any time soon, let’s say in the next century. I admit that the case against the lifting of chip controls is stronger under B), though I am ultimately inclined to reject it in both scenarios.

Why is easing of chip controls, imho, a good idea if the timeline to superintelligence might be short?

If superintelligence is around the corner, here is what should be done: an immediate international pause of AI development until we figure out how to proceed.

Competitive pressures and resulting prisoner’s dilemmas have been identified as the factor that might push us toward NOT pausing even when it would be widely recognized that the likely outcome of continuing is dire.

There are various relevant forms of competition, but plausibly the most important is that between the US and China. In order to reduce competitive dynamics and thus prepare the ground for a cooperative pause, it is important to build trust between the parties and beware of steps that are hostile, especially in domains touching AI.

Controls make sense only if you are very confident that superintelligence developed in the US, or perhaps in liberal democracy more generally, is going to turn out well for humanity, and you are not so confident about superintelligence developed in China.

I do think that the US political system is, in an important sense, whose exposition would require more than an oversized quick take, better than the Chinese political system, but despite this, I am not at all confident that superintelligence developed in a liberal democracy would be good for humanity; frankly, I find such beliefs wholly unfounded.

Why is easing of chip controls a good idea if the timeline to superintelligence is long?

Disclaimer: I know following argument is perhaps too abstract and I am even less confident in it than in previous section, nevertheless.

It is a feature of the current international order that there is a strong taboo on military action of one country against another outside narrow and contested exceptions. I think this taboo is very good and very important. And yes, I know that this taboo is sometimes broken; that does not invalidate previous two sentences.

Another feature of the current international order is that some countries are what is sometimes called pariah states; I am vaguely aware that the word “pariah” has unpleasant connotations in the Indian cultural context, but I am going to use the term anyway, since I don’t have a suitable replacement, sorry. Anyway, good examples are, e.g., Russia or Afghanistan.

Being a pariah state means essentially that some hostile actions which are considered beyond the pale against a non-pariah state are considered ok against them. Notably, the taboo against direct military action is so strong that it mostly applies even to attacks against pariah states, but certain types of heavy restrictions on trade and cross-border investment are what essentially define pariah status. Another sometimes used action against a pariah state is external support, including by providing arms, to an internal opposition, but that is facultative. Many pariah states don’t have internal opposition capable of or interested in armed resistance.

In trade, pariah status tended to be a package deal. If a country is a pariah, it usually had heavy restrictions placed on its trade and investment; if it is a normal member of the international community, it could expect that it would not have certain kinds of restrictions placed on it.

Of course, the previous paragraph has to be immediately qualified in that there are many, many types of restrictions on international trade considered completely acceptable even among normal members of the international community, such as tariffs, bans on investment in critical infrastructure, or bans on selling some military or military-adjacent technologies. But there are other kinds of restraints that were usually deployed only against pariahs.

And pariah status is not a category enshrined in international law; really, making and unmaking a pariah is based on a decision of the government of the United States, since the US is the current hegemon of the international order. E.g., Cuba is, imho, a pariah state mainly because of random historical contingency plus quirks of the US electoral system. BUT breaking a taboo against militarily attacking another country is the sort of thing that tends to get you on the pariah list.

This all means that avoiding a pariah designation is a strong incentive for countries NOT to engage in military aggression.

Now, I do think that the chip controls imposed on China by the Biden administration are breaking this logic. These controls are restrictions on selling a critical component of consumer technology which is related to military stuff only in the sense that, like, steel or oil is related to military stuff. And restrictions on selling those is something which has not usually been done against non-pariah states; the Biden administration, however, had no plan to impose a whole package of pariah status on China. They made an awkward exception in an important area, which devalues the status of being non-pariah member of international community.

Thus, the imposition of chip controls reduces the incentive for China, and indirectly for everyone else, to avoid pariah status. Which means that it reduces their incentive to avoid military aggression. I think that that is very bad and it outweighs benefits of those controls even in no-near-term-superintelligence-threat world.