A response to Gergő and Aaron’s recent essays on context and worldview.
My Perspective on the Disconnect
As someone who’s been working to transition to AI safety for the past 6 months, I’m quite familiar with the context & worldview gaps that Gergő and Aaron both wrote about recently. If you’re assuming I intend to argue against their importance you’re mistaken – the gaps are real. They are two sides of the same disconnect, and this disconnect is a far bigger challenge for the AI safety community than it may appear to be.
When I first began exploring opportunities, I wasn’t aware how large the disconnect was. I saw all the “we need talent” messaging, and, with a very successful background helping employers find talent, I assumed it would be easy to engage and join an org looking to scale.
Rejection does a funny thing to someone like me. I’m competitive, I’m a problem-solver, and I don’t quit on things that I care about. And although I came in with a casual understanding, I quickly became consumed by the AI safety mission and committed to the cause. After not getting any interviews and even being rejected for the BlueDot AGI course, I decided to sunset my business and go all in on learning about AI safety. I stopped sending applications into the void and spent my time reading, writing, and thinking about AI risks as if it were a full-time job.
After almost 3 months, I reapplied to BlueDot with a much stronger foundation of understanding. This time I was accepted, and since my initial rejection led me to assume that everyone in the course would be very knowledgeable, I looked forward to connecting with others that would match the conviction and concern I had developed over the past few months.
I couldn’t have been more wrong.
The people in my group were all sharp and seemed highly capable, but they weren’t anywhere near as concerned as I was. While I came in having already deeply engaged with many of the readings, it seemed as though they were being exposed to some of these ideas for the first time.
I had found myself on the other side of the worldview gap.
Unit 3, the one on pathways to harm, was where the disconnect hit me the hardest, and I walked away from that discussion feeling more than a little disoriented. Since I’m the type that processes through writing, that’s exactly what I did. I won’t share the full essay I wrote immediately after the session, but I will share some of the comments from my peers that led to my disorientation:
“I don’t think authoritarianism is a pressing concern.”
“These scenarios are depressing. I’d rather be optimistic.”
“I can see how things could go bad, but I don’t think that the people in charge will let that happen.”
“The technology isn’t good enough for that to actually be possible, and even if it was I don’t think that would qualify as an existential risk.”
“I don’t believe we should worry about that, but I didn’t do all the readings so maybe I’m missing something.”
Obviously my experience isn’t reflective of everyone who participates, but it left me craving a level of engagement and understanding that I simply wasn’t able to find among the peers I was exposed to. Since then, things have changed a bit on my end as I’ve shifted my focus from only learning to now intentionally seeking out that kind of engagement. I've connected with SPAR fellows working on projects where my expertise is relevant and have been sharing some of my writing on Substack, both of which have been far more satisfying than filling out job applications. But I’m not posting this here to give an update on my journey.
I’m posting it because the disconnect from these gaps is far more consequential for those already in AI safety than it is for the people trying to work in AI safety.
The Larger Consequences
These same gaps that keep qualified people out of AI safety orgs are what keep AI safety from getting support from the rest of the world. The number of people who think about AI risks regularly is relatively small, and the vast majority are already working in the field or actively trying to enter it. That’s not to say members of the general public don’t have opinions on AI risks, they certainly do, but that hasn’t translated to sustained attention and engagement, and when it has, their focus centers on matters far removed from the priorities of the AI safety community.
AI safety needs to find new ways to engage with the public and build support for the cause.
This might be hard to recognize for many within AI safety orgs that are focused on doing good work and don’t feel the need to adjust their approach. As someone coming from the business world, I can’t help but look at the circumstances through a different perspective. You could build the best product and still see your business fail if you don’t know how to sell it. It doesn’t matter how good the research may be or how accurate the predictions are if the work doesn’t translate to meaningful influence on the broader power landscape. Through that lens, AI safety is failing.
We are currently on the Plan D pathway, and we’re ahead of where the top AI safety forecasters predicted we’d be at this point. RSI is either here or around the corner, and we could be looking at an intelligence explosion very soon. And while adoption may be slower than anticipated, that does little to curtail the risks we may be facing on the path ahead.
The frontier labs aren’t listening. The government undermines AI safety and calls AI risks a hoax. Media coverage centers on scandals and controversies, doing little to further the cause. The public is largely indifferent and focused on their individual lives, and those who aren’t are far more concerned about job displacement or the idea that “data centers waste water” than the existential risks. Waiting for another warning shot and hoping it draws people into the cause is not a viable strategy. AI safety needs to go on the offensive, and it needs to start sooner rather than later.
To be clear, I’m not saying the community needs to change what it focuses on. Keep doing the work. But if you want that work to be as meaningful as possible, new approaches to engaging with the public should be considered, and individuals that can drive those efforts should be welcomed. The field has spent considerable effort trying to get people outside of AI safety to understand its worldview, when what it should be doing is figuring out how to frame and deliver that worldview so it may actually influence the direction we are headed.
I’ve spent the past handful of weeks immersed in ideas related to that, and I have some strategies in mind that I believe could position AI safety to more successfully raise awareness, build support, and drive meaningful change. I’ll save those for another post, but I’d love to connect with anyone else who recognizes this problem and wants to get involved or discuss what I’ve been working on.
You can message me here, on Substack, or send me an email.
Good read! Your argument that the same disconnect keeping experienced people out of the field is also limiting its ability to engage with the wider world really resonated. Where I'd push a bit differently: you frame the challenge as how AI safety frames and delivers its worldview so it gains influence. I think part of the gap is that there's little opportunity for people outside the field (including those with no interest in joining) to help shape a future transformed by AI.
I recently applied to the Kairos Generator Residency. I proposed building what I've called gradual empowerment systems — ways to give ordinary people a meaningful say in how AI is adopted and embedded into every facet of our lives. Because, even if we avoid catastrophic outcomes, the arrival of ASI (or maybe just AGI?) will fundamentally change how we live, work and organise society.
There are hugely important questions about what kind of future people actually want, and what trade-offs they're willing to accept. How should the economic gains from AI be distributed? What happens to people whose work is automated? How much human involvement do we want to maintain in healthcare, education and public services? How much privacy, autonomy or human connection are we willing to trade for greater efficiency and convenience?
These aren't exclusively technical questions for AI researchers or policy experts. They're questions about people's values, preferences and lived experiences. And the answers shouldn't be determined solely by a handful of technology companies, policymakers and experts.
My idea is to move beyond persuading the public toward our priorities, and more towards helping people understand possible futures and policy choices so they can weigh up the options and have their preferences fed into real decisions. The pitch to Kairos is that this is another form of field-building, but without that being the primary goal. It could help create a broader base of informed citizens, supporters, and future contributors, while bringing perspectives into AI safety that might otherwise be missed.
Humans tend to overlook extreme risk scenarios that seem far away or even unlikely (whether this is true or not).
AI Safety should also focus communications on risks and dangers that are already present, even if less extreme (cyberattacks, concentration of power, disrupting critical national infrastructure, etc.).
If the field keeps seeming to the outside world as a "science fiction / doomers gloomers", nothing will change until maybe is too late. No one wants to constantly hear about extinction or doom. Make people worry about money in their banks accounts, how national health equipment could fail, or disruptions in telecom systems in planes rising accident rates. And how increasingly powerful AI relates to all this.
Animal Welfare is not only focused on eliminating animal consumption, but also about smaller battle victories such as ending animal factories, that gradually lead to the same outcome. AI Safety should learn a lot from this approach.
Spot on. I agree with your points, and ultimately they can keep working on x-risks while using more immediate threats as a way to build awareness and support. They don't need to change where they apply their focus, but messaging around more accessible entry points for public engagement could be a great move.
However, my ideas go beyond just fear-based messaging. That's one way to get buy-in, but it's not the only way, and I think there are other approaches worth exploring. That's a big part of what I've been thinking about and the org concept I'm working on.
Glad to hear that, Karim. Can I ask for that new organization concept you are working on? I'm also curious that you are deciding about founding something instead of applying for new roles after the conext you have gathered.
This post does a great job of highlighting why I'm compelled to start an org, and the TLDR summarizes it really nicely: Figure out what needs doing and do it, don't wait on approval from fellowships or jobs. I see gaps, I'm confident I can fill them, and I'd rather use my capabilities to contribute than continue trying to demonstrate that I have those capabilities in the first place.
I've also learned far too much about the hiring practices of AI safety orgs to justify the cost-benefit of pursuing roles through applications. Most orgs don't have adequate processes in place to evaluate inbound talent, so they lean heavily on referrals and candidates whose alignment is easily recognized. Gergő and Aaron highlight how much they value context and worldview, but neither address the difficulty of identifying outsiders who have developed those qualities. There's also a substantial opportunity cost if that alignment is weighted more heavily than deep functional expertise, which I've expanded on previously in comments on other posts about this subject.
The org concept I'm working on ties into what I wrote about in this post: new ways to engage with the public and drive support for the AI safety cause. There's much more to it than just that, but the primary outcome I'd be looking to achieve is to make AI safety a priority for everyone rather than just those working within AI safety. I'd also look to hire heavily from the talent pipeline rather than pull from other orgs, and I think that's an approach that should be welcomed considering the staggering mismatch between capable external talent and available opportunities for them.
It seems reasonable to me that courses designed to introduce people to AI safety would attract participants encountering some of these ideas for the first time. That's what the courses are there for, right?
Your coursemates being less alarmed also doesn't mean they were less informed. Even people working at the frontier labs seem to disagree about how concerned we should be.
But the question I'm itching to ask is - given that you seem to have acquired the kind of context that the earlier post identified as a missing ingredient among candidates, and given your extraordinary efforts over a prolonged period ( that is not sarcasm; you really do seem to be living and breathing this topic), have you managed to secure a role in AI safety yet?
I definitely see your perspective! Getting rejected from the course the first time I applied (back in June) largely influenced my assumption that I needed to strengthen my understanding if I wanted to get in. For all I know, I could have applied a week later and been accepted back then, and had I done so I'm sure the experience would have been very different.
On coursemates being less alarmed, I agree. A lower level of concern doesn't necessarily mean someone is less informed. I'm sure there are plenty of people that know far more than I do and simply disagree with my perspectives. That said, in the discussions I participated in, many of my peers shared that this was their first time encountering some of the arguments or even that they hadn't done all of the readings. That's not a knock on them - many are still working full-time roles, and there's a lot of literature to engage with in order to develop a solid foundation of understanding. Not everyone can stop what they are doing and commit full-time to learning about a field they want to enter, and, to me, that's largely why these context/worldview gaps exist in the first place.
And no, I haven't secured a role, but I also haven't applied to any since September. My focus has shifted toward developing a concept for a new organization aimed at addressing some of the gaps I see in the field. The types of roles I was initially targeting are fairly senior level and relatively few and far between, and I think there are plenty of meaningful areas of impact that a new org could pursue.