I'm sort of mid-transition towards becoming vegan, and frequently the issue of parity does affect me. A lot of times, I'm simply hungry for something that tastes like the food I've eaten to relax for decades. The palette of things that I can readily eat that are vegan is narrow, and a lot of them aren't that great. And I know several people who have been vegan for many years, and when I look at their diets, it seems like frankly many of them do not really relish cooking or food at all. That's anecdata, but I do not think it's true that a vegan diet at parity with the non-vegan culinary traditions is readily available and/or that the public is well-informed about it and just resisting in an attempt to suppress cognitive dissonance.

Just yesterday, I had an Impossible Bratwurst with sauerkraut at Sunflower Bakehouse, and it was amazing. I could see myself ordering it again and again. For once, I felt kind of excited, like my nascent veganism had led me to something I actually enjoyed eating, something I wanted to tell my friends about. This is the first and only time I've felt this way about a vegan food product.

Food in contemporary society is addictive. I have daily impulses to go to the local coffeeshop and get a latte and a bagel, even if I'm not hungry and I'm fully awake and have a nice espresso machine at home. I'm not sure what sort of psychosocial-conditioning woo is going on at Momento Coffee in Hermitage, TN but our consumeristic relation to food creates a highly competitive market, and my vegan friends enter into that fray with a bottle of Soylent and a banana.

I think people will easily overcome cognitive dissonance to eat something they find tasty, so I don't buy that legumes are really yummy but people are resisting them because of internal conflict over their self-representation, especially when switching to them would resolve that internal conflict. Most people I know feel no conflict at all—they never think about the animals—so presumably their non-interest in legumes is that they find them comparably less enjoyable to eat.

I see in a subsequent comment you bring up the banning of cell-cultured meat. I think agricultural special interests are mostly driving this, not οἱ πολλοί, but inasmuch as the average person wouldn't eat a cell-cultured burger, its probably because they associate it with stainless steel vats and men in white coats holding vials of mysterious fluids. There is a huge market for "organic" meat and produce, which is driven by very different concerns than suppression of cognitive dissonance.

I think it's the combination of framing the roles as being desperate to hire while not being straightforward about what the requirements are that feels bad. And to be clear, it's not so much the technical requirements which are underspecified, though those can be, its probably more the soft requirements. If you discover EA online through personal ethical development you might be mistaken into thinking it is less elite and thus less credentialist than it really is.

Other comments replying to this post somehow give off the connotation that getting into Harvard or Google isn't that big of a deal, or that if you fail one of those, you probably will get into another FAANG or Ivy. To be clear, I'm not maligning those commenters, just noting how even passing comments on quick takes seem to come from a different universe of achievement and social connection than myself and my imagined mid-career-transitioners that this post is about.

If you are reading this and you recognize that you wouldn't get into an Ivy league school or a FAANG role, that should give you pause when considering doing a career transition to an EA org. If you want to talk about how to maintain your ethical values and your desire to live a high impact life given your background, I'm happy to lend an ear. To be clear, I'm not saying it is impossible, but, given that your time and energy is valuable and finite, I would encourage you not to be naïve about it.

These orgs have a high talent bar and only a few open positions in absolute terms. If you don't have the right educational or career background, you won't be legible to the resource constrained hiring teams. They are going to go with who they know. It's only rational.

I would encourage you to (a) try to build some local community/friendships of people who share your values, (b) consider earn-to-give, and (c) recognize that if you want to do direct work, it is probably a long-game for you. I decided it wasn't to my comparative advantage. But I have friends to talk to, I did change jobs to something I like more, and I can look in the mirror and say I'm doing about as good as I think I can today.

On the hiring front, communicating number of applicants is good. I also think just being explicit, like listing what level of education is expected (go GiveWell for making clear the junior researcher role expects a Master's Degree back in January), how many applicants applied, would be helpful. Staff bios on the site are also helpful TBH. I scroll those things, see where people went to school or worked, and quickly get a sense of whether it's worth my time to apply.

So, before I go any further, I want to state that personally, my AGI timelines have updated somewhat, to basically thinking AGI is here. At least, at the time of our discussion, I hadn't really processed the implications of AI agent swarms. Even if any given singular model instance fails your personal test of AGI, and even if the swarms are still pretty jagged in terms of intelligence, at this point, it just seems silly to not acknowledge that AI agent swarms are intelligent enough entities to pose a serious threat to us human beings at say, the level of a cyber-hacking group at the very least. Right now, swarms are still very new and very expensive and so they are not heavily deployed, but if they achieve adoption levels similar to say, Claude Code, we could have no AI advances at all ever again, and the world would look radically different in short order.

Further, my p(doom) has mostly gone up, because I suspect just slight improvements in swarm social architecture and scale would constitute Bostromian collective super intelligence, and this is frankly faster than I thought we'd get here, so I don't think we are very prepared at all. Sorry if that all was a bit incoherent, but I say that so that what follows is not misunderstood as indicating some sort of doubt about AI progress.

OK, thing one, less important. It seems like the swarm system functionally results in "higher intelligence." I don't know if this should count as a point in my favor for my earlier claim about the harness mattering a lot relative to the model. But it does seem like the swarm architecture is part of the story, even as the model intelligence continues to rapidly increase. Honestly, it seems like they are just different components and so can't exactly be compared against each other. High model intelligence matters a lot even as the ability to coordinate 10,000 of them at once sort of compensates for things like limited context windows and a lack of deep persistence. Individual sessions are very smart in terms of serial thinking power, but when we think of an extended Turing test, we really need more things than just raw reasoning horsepower, we need contextual awareness, large-scale goals (big enough to in turn lead to developing instrumental goals which final goals tend to converge on), deliberation that leads to something like in-context learning. The swarm architecture allows for these other things to happen, despite models never themselves developing continual learning or astronomical context windows.

Secondly, I think I'm a bit vindicated in my specific claims relative to Mythos and cybersecurity. Specifically, in Anthropic's latest report, the section on cybersecurity details numerous impressive automations that probably are basically what people were imagining when Mythos was dropped... And they all were done by Opus or lower class models. To be a bit clearer and more fair, I suppose what I'm saying is that the cybersecurity nightmare is fully within reach without Mythos-class intelligence, though it would be reasonable to respond by saying, "With Mythos-class intelligence, it would have been another OOM more terrifying." And I agree with that I guess.

To be specific, I'm especially thinking of GTG-10007, which details the creation of exploit foundries. Here we see less-than-Mythos class intelligence being harnessed systematically to create novel exploits, which is kind of what I was saying I was worried about. It just hadn't become public news at the time of the Mythos announcement.

Lastly, I've basically changed sides on hype discourse. Its not that I think there aren't people who are overstating or misstating things. The recent OpenAI hugging face attack had many re-tellings before landing on what I hope is the final and true one. Listening back to the earlier re-tellings, such as that the agents were trying to find the answers, makes me cringe a bit. Apparently, to some degree, the various speakers in those instances were reporting false speculation as fact, knowingly or not. But I guess I now feel more forgiving about such things because my personal sense of alarm has shot through the roof somewhat irrationally late.

Thank you for jumping in and joining the discussion! I'm particularly excited to read more into the Berkeley Vulnerability Initiative, just to understand the project's larger goals and techniques.

I think, as per other comments, that determining whether something is hyped or not is pretty subjective, but I do feel that my main prediction, that carefully scaffolded harnesses and systems would be reliably pumping out Mythos-class vulnerability discoveries by the end of the year, is pretty clearly already true.

I do think its clear that progress in AI offensive cyber capabilities is a real and present concern—I find myself actively worrying about the digital money in my bank accounts—so, I would consider that a valid takeaway from Mythos and subsequent developments. We are clearly very close to a situation where AI progress could endanger a lot of our digital infrastructure, if it just had the funding and someone with bad motivations to push it. That's a big change from how I perceived the world one year ago, and seems worth being excited about it.

It also, to the technical point of this discussion, seems to be a capability gain that is gated on model intelligence, or at least, that's how it looks to someone like me, with a naïve picture of the current state of model training/development.

Question from a newbie. I am constantly seeing negative references to the gutting of US foreign aid. It seems pretty clear that global development-focused EAs generally view the change in policy to be a bad thing. But I do not think I have once seen any discussion at all about how to reverse this state of affairs. Building on a running theme as of recently, it seems like political giving may have an outsize effectiveness, due to the relatively sparse funding in the space. So, naively, it would seem like you probably could get a great rate of return on efforts to reinstate USAID. I understand that political coalition building and organizing is not easy, etc. I'm not someone with those skills, just a rando. But I'm a little surprised that I don't think I've ever seen it taken up here when it seems like the downstream effects are making our goals harder to achieve. Basically, not only is it at face value cost effective, but also, we are collectively burning a lot of human capital working around this problem. Why not confront it head on?

I agree with you two. I don't have any delusions about avoiding all risk of causing harm or that harm avoidance is straightforwardly more important than providing benefits. I guess what I am saying is that risk of causing harm is distinct from risk this is ineffective, and it would be nice to see these broken out (as someone who works in data analytics and engineering, I realize the real world is not so simple). It seems to me that actively causing harm is a bit of a different thing than just being ineffective, and you would ideally reason about it specifically, rather than bundling it all together into one big effectiveness metric.

Especially since, while I don't think we should try to avoid all harm, people may have different moral weights about causing harm. For some people, they may be much more indifferent about causing harm relative to providing some benefit, whereas others may have a stronger bias towards "first, do no harm." Given that the tradeoff between these is something each individual must determine, it is better to separate it out in your model and allow people to discount the effectiveness according to their own priorities.