P(doom)

At the end of my previous post here I said "Don't hold your breath. The AI revolution ain't gonna happen this way." I think that there are some pretty good reasons to think that AGI/ASI may be further away than some (eg Altman et al) would like us to believe, especially following what's gone on in the last week or so regarding DeepSeek. I'm going to post something here about that shortly, once I've done a little more thinking and I feel inspired by the muse. But before I do that, I wanted to write a little more about the existential risk posed by AGI/ASI, even if it isn't likely to be coming in the next 18 months. Nothing like a bit more gloom and doom at the end of January, eh?!

P(doom) is a term in AI research that refers, simply, to the probability of a catastrophic outcome ("doom") for the human race resulting from AGI/ASI. In a 2023 survey, AI researchers were asked to estimate the probability that future AI advancements could lead to human extinction or similarly severe and permanent disempowerment within the next 100 years (Wikipedia). Some of the responses given by prominent AI researchers and others in the field are shown in the table below (from Wikipedia, with highlights added by me). Link to original in references below.

Notice anything? Notice how the tech bros¹ (blue boxes) are noticeably much more bullish about the future with AGI/ASI than the mass of the AI commentators (red boxes) - who, coincidentally, don't have a dog in this fight? Others (purple boxes) are more on the fence.

Say no more, squire - as Eric Idle would say (Nudge Nudge).

[Edit, to add (3 hours after uploading original).] OK, my original final sentence feels flippant now. But I'll leave it in. Just found this on X, in a repost from a few days ago by Eliezer Yudkowsky:

And finally, look at the difference between expert predictions of when HLMI will be achieved from two surveys of experts (N = several thousands) in 2022 and 2023, both in terms of timing and in terms of uncertainty.

Blimey 😨

Footnote

1. Someone, usually a man, who works in the digital technology industry, especially in the United States, and is sometimes thought to not have good social skills and to be too confident about their own ability (Cambridge Dictionary).

References
P(doom) - Wikipedia
Tech bro (definition) - Cambridge Dictionary
Nudge Nudge Monty Python - YouTube
2023 Expert Survey on Progress in AI - AI Impacts Wiki