But he has many followers who see him as some kind of one-of-a-kind world-historical genius, some kind of nearly infallible mystical guru figure.

I'd be surprised if it's as many people as you think. Most rationalists I've talked to think Eliezer has a bit too much ego and that he is a bit too confident about many of his predictions.

For what it's worth, I think regardless of whether he ends up being right or wrong, at this stage he's clearly a world-historical figure. I think he is a genius, but I also think that someone being a genius isn't as valauble as it's often portrayed. Intelligence is not the same as wisdom and, as a result, someone's extraordinary intelligence can end up being a liability, rather than an asset.

Strong disagree. A language is a way of life, not a set of memorised answers. I agree that it would be nice if there was some more honesty about what exactly talent-constrained means. I like how this post clarifies it.

It seems easier to compile a syllabus "here is what you need to read to have a good overview over the strategic picture on AI safety" compared to "here is what you need to read to become a good grantmaker".

Agreed, but I suspect the former is harder than you might expect. Much of the material you would need to make some high context doesn't exist and, in any case, I don't know if it's as easy as just "read this syllabus". To quote Wittgenstein, "a language is a way of life".

Very excited to see that someone actually did this!

I've been saying for a while that someone really need to run a course based on Effective Altruism in the Age of AGI. For many people, a traditional EA Intro Fellowship isn't going to resonate given how fast AI is moving.

In case it's relevant for the future, I have some notes on take-off speeds and RSI (I'll share Additional Context as well even though there's a lot of overlap since I copeied some material from it and I haven't had time to reduce the overlap).

In terms of potential themed weeks:
• Responses to Effective altruism in the age of AGI
• Has EA been a positive influence for AI safety? Various people (ie. Michael Nielsen, Richard Ngo, ect.) have made arguments that suggest EA's influence might not have been positive.
• What should EA make of the Integral Altruism movement? Are their critiques correct or would shifting in this direction undermine what has made Effective Altruism successful? (alternatively, we could look at Rutger Bregmen's Moral Ambition)
• PR and polarisation: Is EA being careless with its reputation or is the PR mindset toxic? Should EA engage more with the right (in light of polarisation against us) or should we be more confrontational?

Some people have recently been talking about a decrease in activity on the forum. I've written before about the possibility of a yearly theme. I don't know if it would work, but it seems like it might be worth a shot to counter the risk of stasis. It's somewhat hard to articulate why I think things like this might be valuable, but one of the barriers that a community can face in terms of moving forward is that there's so many potential different topics that people could talk about. Purely relying on the marketplace of ideas often means that it's hard to build momentum around any particular direction the community could be exploring. Now maybe running weeks focused on particular topics already covers this and does it better, but I think it's at least worth exploring.

I don't know very much about Wytham Abbey (and I've never been there).

But it seems like the complaints about a conflict of interest and "EA is wasting money on castles" actually ended up distracting from what seems to have been the more important lesson here, which is about domain knowledge: something can seem well-suited to a particular use case in the abstract, but may turn out to be poorly suited in the concrete. No-one had any interest in buying a building poorly suited to the use case and, if anything, this is the kind of mistake that a closer relationship between a funder and a grantee might actually be able reduce by involving the grantmaker at an earlier stage of the process. Now there are risks as well, as you point out, but these can be mitigated (I'd probably feel more nervous about the CEO of a large funder sitting on the board of an organisation they were funding, not saying I'd be completely against this, but they wouldn't have a direct boss to evaluate them. There'd still be the board, but an employee would have two potential lines of oversight rather than one).

"So the onus is on you to justify your opinion that these processes miss "real virtue or integrity" — I don't agree. I think we should take a more nuanced stance on how these norms evolved. Effectiveness is obviously a part of it, but it's only one component. Have you heard of the book Seeing as a State? States focus on legible metrics like COI because that is the mechanism that is available to them and because it reduces the chance they can be blamed if things go wrong. I'm not saying that these laws should be repealed nor am I saying that concerns about conflicts of interest are minor and can be waved away. I'm just saying that these shouldn't colonise our minds and I think you're giving them too much deference in terms of how we should handle situations where there isn't a legally mandate^[1]. Nor am I saying that we should ignore social norms either—integrity is about right relationship with society and yourself. It's about orienting to both.

"When easy mitigation was at hand" — except if the purpose of her being there was to help ensure that the money was spend well...

1. ^{^}

   Just to be clear, this isn't an endorsement of looking for loopholes in the law. I'm only talking about situations where the law doesn't apply and it isn't due to an obvious loophole.

There are many ways in which EA could have handled it better and I agree with your headline: "EA isn't immune to money, status and power".

At the same time, I really want to push back against the simplistic narrative that Wytham Abbey was obviously a silly and ineffective thing to buy^[1]. Lots of people were skeptical that £14.9 million pounds could possibly be well spent buying a historic building when it could have saved like 4000 lives based on GiveWell figures. I think it should be quite clear today that this was way too dismissive.

EA correctly called AI as one of the largest issues of our time early and, by taking this gamble, has managed to achieve dramatic influence over the way this is playing out. This grant was a bet on purchasing a nice venue for retreats could allow EA ideas to become more influential. Given how influential EA ideas have become subsequently, it feels plausible that this kind of bet could have worked out. There's definitely a part of me that views this way of influencing the world as shallow, but, at the end of the day, the world is how it is, and you can have a lot of impact if you can bring the right people together.

Now, it may turn out that the influence of EA on AI is actually net-negative. I put a decent amount of weight on this possibility. At the same time, I mostly don't think the folks criticising the castle should get to claim this as a victory^[2]. On the contrary, I consider the whole castle brouhaha to have been a distraction from more substantial issues in terms of integrity.

Many people didn't necessarily object to the castle in and of itself, but more as a PR risk. I think Will MacAskill does a great job of explaining why the "PR mindset" was likely a mistake: "Post-FTX, I think core EA adopted a “PR mentality” that (i) has been a failure on its own terms and (ii) is corrosive to EA’s soul".

It wasn't obvious at the time, but it seems pretty clear to me now that a lot of EA's post-FTX reputational issues were actually the result of EA-on-EA fire and then various folks or groups being unwilling to sufficiently back themselves (if you're not willing to back yourself, why would anyone else back you?) rather than anything that happened. Honestly, I think that a lot of the infighting that occurred after FTX was low integrity behaviour, in particular, people taking advantage of the moment to litigate their grievances. Now if this is true, and I think it is, it creates a bit of a bind. On one hand, criticising the motives of critics most typically ends up sliding into a group becoming more insular and closed-minded. On the other hand, giving bad faith critics free reign without ever calling out their bullshit can be disasterous too, as the near-collapse of EA demonstrated. After all, who wants to be part of a community if there's an active effort to tar the community's reputation, the community itself is full of drama and almost no-one is sticking up for people who are fighting their hardest to make the world a better place?

Just to be clear, there are extremely strong reasons be wary of pushing back against criticism. It's very common for this to start off reasonably and then escalate. At the same time, reversed stupidity is not intelligence. Just because the default is for communities to mess up their epistemics by being overly reactive against critics, doesn't mean that you can swing all the way to the opposite end and everything will be fine and gandy. For a long time, it worked rather well, but then the community obtained money and power and the wrong people flooded in and then when things blew up, they took the opportunity to litigate their petty grievances and the EA community that had previously been able to operate on an assumption of good faith was mostly caught flatfooted and let things play out by default, when we needed to be defending ourselves. (Lucky for us most of the people with the wrong motivations left shortly after).

You linked to your comment here as an example of EA doing poorly on conflicts of interest. Honestly, I think a) you're wrong and that Claire had an extremely reasonable explanation b) your criticism focused on prioritised non-mandatory process over the deeper principles at stake c) I honestly think you should apologise to her and admit you were wrong d) maybe there's enough uncertainty that an apology wouldn't be justified, but I still think it's at least somewhat arrogant for you to be acting as though you were clearly in the right here.

Just to be clear, I'm not trying to put you in the crowd of "wrong people" or "bad faith critics", you seem pretty good faith in this post and I can't remember if I've read any of your other comments or posts so I'm going off this post.

But from what I've read, I suspect that you want the community to focus a bit too much on process over real virtue or integrity. Just to be clear, I'm not trying to dismiss the importance of rules, you can follow almost all the rules, but then let one slide and create major legal trouble for yourself and your organisation. Additionally, there are certain norms that are important, not so much because of legal reasons, but because not following them would be a serious breach of trust^[3].

My position is simply that I don't think we should take this so far that we let processes colonise our minds as to what true integrity looks like. If money, status and power draw the wrong people to the community then adding just a tiny bit of extra process around the margins isn't going to save you (but it may distract you).

Don't get me wrong, I think what you're doing here is great (honorable even)!

Accordingly, I am deliberately selecting Board members with a specific brief to be ethical watchdogs within the organisation. I have approached people I believe to be unusually high integrity, and unusually willing to challenge or confront me if necessary.

Second, I am asking some people to become personal ethics advisors to me, completely separate from the Board. This is because I think the bar for someone to approach a Board member with feedback is quite high. I think that external ethics advisors might be able to receive more feedback, including “epistemically rigorous gossip”, than charity directors. They might also be perceived as more impartial.

So where do I disagree:

a) "These are easier to mitigate, because we can choose to change our own behaviours more easily than we can gatekeep the community" — maybe we should avoid solutions closer to gatekeeping because they would be infeasible or they would backfire, but I certainly wouldn't make a decision here merely on the basis of one path being "easier".
b) "If you prefer to appoint people you trust personally and who share EA values to positions of responsibility, they will be conflicted or appear to be conflicted more often than in bigger communities" — We should avoid these conflicts when we can; own it when we can't or when the cost would be too high. Again, we need to comply with all relevant laws, mistakes here can be very costly, but it's worth understanding that laws are designed for a low trust environment (ie. all of society). We can't let this completely colonise our mindset. Some people want us to shift much further towards low trust norms, I feel this is defeatist and so I oppose.

1. ^{^}

   It may have been a bad investment if it were underutilised or if it didn't deliver significant value over hiring external venues.

2. ^{^}

   I said "mostly" because this depends on the specifics of what they said.

3. ^{^}

   I haven't thought deeply about this yet, but I wonder whether thinking about whether an action would be a "breach of trust" with the public would be a healthier mindset than PR mindset but I haven't thought deeply about it. Arguably, the recommended PR response to many situations would be at least a minor breach of trust because you're optimising your interaction with the person to obtain an outcome rather than genuinely engaging with them.

I think it's less true now if you buy short timelines, but I've often thought that the EA ecosystem underrates internships as internships produce positive externalities for the ecosystem as a whole.

I really like the table.

That said, putting SPAR and AIS camp in the same box as ERA is wild.

As a rough mental model, there's the part-time unpaid AIS programs, the full-time paid competitive fellowships and then the more elite-tier fellowships.