FYI this is currently called "Untitled Draft" :) 

This resonates with me, thanks for sharing. 

I suspect you're a lot less alone on this one than you are with donating to effective charities and were being early to COVID. (I also suspect you agree, but spelling this out for other readers).

It's tricky to get meaningful information from surveys, but this one from June finds that a slight majority of respondents thought a superintelligence would seek to take control of humanity. AI existential risk awareness is also steadily climbing, this survey says up to 34%. I would bet that still only a very small % of people are as worried as you and me and many in this community, but lots of people are somewhat worried and could probably quite quickly get more worried and be activated to do something about it.

I think this points to saying what we're worried about, and saying it clearly (like this!), especially how quickly loss of control risks are becoming very real. I'm very bullish on AI safety comms, and expect there's lots we can do there.

I just stumbled on this post. The recent OpenAI<>Hugging Face incident (and wave of related security incidents) seem to be what this post described ("a non-existential public display of misalignment"). 

It seems like lots of people in the AIS community were responsive to these incidents, but we might be in an even better position now if more people read this post. Thanks, keep posting!

Excited for this event!

Can I subscribe just to a newsletter for occasional calls to action? The "Join" form asks me to agree to the volunteer agreement, which I don't want to do (e.g. I don't want to give PausAI UK permission to use my name in materials), but maybe I want to do things like Michelle did occasionally.

There was an update about this here, FYI.

Visions is one my favourites:

Visions
Trying to make sense of the now
Trying to make sense of the past
To show us how

Visions
Imagining the worlds that could be
Shaping a mosaic of fates
For all sentient beings

Visions
Cycles of growth and decay
Cascading chains of events
With no one to praise or blame

Visions
Avoidable suffering and pain
We are patiently inching our way
Toward unreachable utopias

Visions
Enslaved by the forces of nature
Elevated by mindless replicators
Challenged to steer our collective destiny

Visions
Look at the magic of reality
While accepting with all honesty
That we can't know for sure what's next
No we can't know for sure what's next
But that we're in this together
We are here together

We are here together
Ooh, we are here together
Oh, we are here together
We are here together

Jose Gonzalez' last two albums are wonderful and have significant (and explicit) EA influences. He'll be promoting the GWWC pledge on his upcoming tour too!

Thanks Jan, I appreciate the pushback.

Just wanted to flag the group is heavily selected for belief alignment with something like "EA/Constellation/Trajan House" views

As an event focused on x-risk, yes, I think this is fair.

"AI enabled human takeovers" was promoted as agenda to prioritize in multiple widely read memos by high statues people in the community  (which the organisers prioritized in the reading list). 

It's true that:

• The agenda featured some talks emphasising risks from AI-enabled human takeover.
• Some of the most popular memos also emphasised this risk category.
• Some people took the survey after reading the agenda and the memos.

But I don’t think attendees were as strongly influenced as you seem to imply:

• We highlighted some memos to read at the beginning, but soon after launching the memo platform, we prioritized memos by votes from attendees. Memos making the case for more emphasis on risks from aligned AI were heavily upvoted, and some memos that we highlighted from the beginning received fewer upvotes.
• The survey was in part motivated by disagreements I'd heard about how the AIS community was allocating resources. While I'm sure some attendees were influenced by information they recently encountered, many will have thought about these questions in advance of the survey.
• I don't have the full data, but I think it's likely that many attendees completed the survey before engaging with the memos and before the full agenda was published. 

I do think you're pointing to a real effect to be aware of, and thanks for pointing it out, but I don't think it's as significant as you make out (though maybe you don't think it's super significant).

Results are framed as "leaders and key thinkers in the x-risk and AI safety communities agree"

I think the areas of broad consensus accurately (if roughly) reflect the data we have here and what we saw in memos. FWIW, my overall takeaway from running this survey is that leaders and key thinkers have a wide range of views and I think this post captures and conveys this.

I take "AGI goes well" to imply a wealthy and technologically advanced society. I think that could mean:

- Very cheap and delicious meat alternatives.
- Factory farming waning as it reaches inefficiencies and bottlenecks, not able to compete with the above.
- More demand for higher-welfare options like free-range and local produce.

But it also seems possible that we "lock in" factory farming and scale it further and that AGI adopts speciesist views.

Very uncertain, I don't find myself strongly disagreeing with claims across the spectrum.