Hide table of contents

by chaosmage

Building on the recent SSC post Why Doctors Think They’re The Best...

What it feels like for meHow I see others who feel the same
There is controversy on the subject but there shouldn't be because the side I am on is obviously right.They have taken one side in a debate that is unresolved for good reason that they are struggling to understand
I have been studying this carefullyThey preferentially seek out conforming evidence
The arguments for my side make obvious sense, they're almost boring.They're very ready to accept any and all arguments for their side.
The arguments for the opposing side are contradictory, superficial, illogical or debunked.They dismiss arguments for the opposing side at the earliest opportunity.
The people on the opposing side believe these arguments mostly because they are uninformed, have not thought about it enough or are being actively misled by people with bad motives.The flawed way they perceive the opposing side makes them confused about how anyone could be on that side. They resolve that confusion by making strong assumptions that can approach conspiracy theories.

The scientific term for this mismatch is: confirmation bias

What it feels like for meHow I see others who feel the same
My customers/friends/relationships love me, so I am good for them, so I am probably just generally good.They neglect the customers / friends / relationships that did not love them and have left, so they overestimate how good they are.
When customers / friends / relationships switch to me, they tell horror stories of about who I’m replacing for them, so I'm better than those.They don't see the people who are happy with who they have and therefore never become their customers / friends / relationships.

The scientific term for this mismatch is: selection bias

What it feels like for meHow I see others who feel the same
Although I am smart and friendly, people don't listen to me.Although they are smart and friendly, they are hard to understand.
I have a deep understanding of the issue that people are too stupid or too disinterested to come to share.They are failing to communicate their understanding, or to give unambiguous evidence if they even have it.
This lack of being listened to affects several areas of my life, but it is particularly jarring on topics that are very important to me.This bad communication affects all areas of their life, but on the unimportant ones they don't even understand that others don't understand them.

The scientific term for this mismatch is: illusion of transparency

What it feels like for meHow I see others who feel the same
I knew at the time this would not go as planned.They did not predict what was going to happen.
The plan was bad, and we should have realized it.They fail to appreciate how hard prediction is, so the mistake seems more obvious to them than it was.
I knew it was bad, I just didn't say it, for good reasons (e.g. out of politeness or too much trust in those who made the bad plan) or because it is not my responsibility or because nobody listens to me anyway.In order to avoid blame for the seemingly obvious mistake, they are making up excuses.                                                                                                                                                                 

The scientific term for this mismatch is: hindsight bias

What it feels like for meHow I see others who feel the same
I have a good intuition; even decisions I make based on insufficient information tend to turn out to be right.They tend to recall their own successes and forget their own failures, leading to an inflated sense of past success.
I know early on how well certain projects are going to go or how well I will get along with certain people.They make self-fulfilling prophecies that directly influence how much effort they put into a project or relationship.
Compared to others, I am unusually successful in my decisions.They evaluate the decisions of others more level-headedly than their own.
I am therefore comfortable relying on my quick decisions.They therefore overestimate the quality of their decisions.
This is more true for life decisions that are very important to me.Yes, this is more true for life decisions that are very important to them.

The scientific term for this mismatch is: optimism bias

Why this is better than how we usually talk about biases

Communication in abstracts is very hard. (See: Illusion of Transparency: Why No One Understands You) Therefore, it often fails. (See: Explainers Shoot High. Aim Low!) It is hard to even notice communication has failed. (See: Double Illusion of Transparency) Therefore it is hard to appreciate how rarely communication in abstracts actually succeeds.

Rationalists have noticed this. (Example) Scott Alexander uses a lot of concrete examples and that should be a major reason why he’s our best communicator. Eliezer’s Sequences work partly because he uses examples and even fiction to illustrate. But when the rest of us talk about rationality we still mostly talk in abstracts.

For example, this recent video was praised by many for being comparatively approachable. And it does do many things right, such as emphasize and repeat that evidence alone should not generate probabilities, but should only ever update prior probabilities. But it still spends more than half of its runtime displaying mathematical notation that no more than 3% of the population can even read. For the vast majority of people, only the example it uses can possibly “stick”. Yet the video uses its single example as no more than a means for getting to the abstract explanation.

This is a mistake. I believe a video with three to five vivid examples of how to apply Bayes’ Theorem, preferably funny or sexy ones, would leave a much more lasting impression on most people.

Our highly demanding style of communication correctly predicts that LessWrongians are, on average, much smarter, much more STEM-educated and much younger than the general population. You have to be that way to even be able to drink the Kool Aid! This makes us homogeneous, which is probably a big part of what makes LW feel tribal, which is emotionally satisfying. But it leaves most of the world with their bad decisions. We need to be Raising the Sanity Waterline and we can’t do that by continuing to communicate largely in abstracts.

The tables above show one way to do better that does the following.

  • It aims low - merely to help people notice the flaws in their thinking. It will not, and does not need to, enable readers to write scientific papers on the subject.
  • It reduces biases into mismatches between Inside View and Outside View. It lists concrete observations from both views and juxtaposes them.
  • These observations are written in a way that is hopefully general enough for most people to find they match their own experiences.
  • It trusts readers to infer from these juxtaposed observations their own understanding of the phenomena. After all, generalizing over particulars is much easier than integrating generalizations and applying them to particulars. The understanding gained this way will be imprecise, but it has the advantage of actually arriving inside the reader’s mind.
  • It is nearly jargon free; it only names the biases for the benefit of that small minority who might want to learn more.

What do you think about this? Should we communicate more concretely? If so, should we do it in this way or what would you do differently?

Would you like to correct these tables? Would you like to propose more analogous observations or other biases?

Thanks to Simon, miniBill and others for helping with the draft of this post.

 

 

 

This work is licensed under a Creative Commons Attribution 4.0 International License.

Comments11
Sorted by Click to highlight new comments since:

I'm truly confused by this section of the Chapter...Did I miss some pre-reading that I should have done? Why are we talking about communication in abstracts? What are LessWrongians? How does this relate to Effective Altruism? I understand the last few sections of the Chapter and how they fit into Effective Altruism. They're building up an argument about why people don't give to the most effective causes: an inability to care for things at magnitudes of scale and scale insensitivity. I also almost understand the context of the previous section - the TED Talk by Julia Galef. She's saying we need a "Scout Mindset" which is all about curiosity and openness. But this section feels a little out of the blue? Is it meant to make us aware of common cognitive biases...? I suppose so. But then: why list cognitive biases and then jump into why abstract communication often fails? (In a nutshell: I feel like I missed something somewhere! Would appreciate any guidance here!)

Are you going through the CEA Intro Program by any chance? https://www.effectivealtruism.org/courses/introductory-program 

I'd recommend asking these questions to your favourite AI or to the facilitators, my understanding is that course facilitators don't use this forum for various reasons, and all of ChatGPT, Claude, and Gemini are great at answering these questions. E.g. here's ChatGPT:
 

I think you’ve identified a real jump in context, rather than missed some prerequisite reading!

This was originally a post on LessWrong.com, an online community focused on reasoning and decision-making. It does two things: first, illustrates how cognitive biases can feel like reasonable thinking from the inside; second, argues that these concrete illustrations are a better way to teach people about biases than abstract definitions. That second part addresses the original community’s communication habits, which explains the references to “LessWrongians” and makes it feel like a tangent here.

The connection to effective altruism is that trying to help effectively requires assessing our own judgments critically. For example, “I’ve carefully researched this charity and everything supports donating to it” might reflect good research—or selectively seeking favourable evidence. The tables are meant to help us notice that possibility, much as the scout mindset encourages us to examine beliefs we’re attached to. They aren’t proof that any particular belief is biased.

So your interpretation seems right: the main takeaway for this chapter is awareness of cognitive biases. The discussion of communication explains why the author presented them this way. An introductory note distinguishing those two purposes would make its inclusion much clearer.

Hi Lorenzo - thanks for that!

I think the ChatGPT response is good...there's a bit of a jump in context happening here. I think what might help to make this section of the Chapter easier to digest for newbies might be an introductory paragraph that goes something like: "Effective Altruism requires us to think clearly about the world. Behavioural Science has shown that human beings are affected by cognitive biases that many are not even aware of. Becoming aware of these helps us to improve our thinking and decision-making. Here's a post about 5 of these biases and how to spot them: not only in ourselves, but also in others." Something like that.

And yes: I'm joining an intro program from 4 November (2026)! Very excited. :)

The table made it easy to comprehend 

What this article feels like for chaosmage: "I can see how everyone succumbs to the pitfalls of abstract communication and biases like hindsight and selection bias, but I won't fall for it"

How I see this article: "Very funny, guys"

Nice work

I feel I tend towards self-deprecation which is a bias unconsidered, but that is just because I have depression and am abnormal in this respect to most (this statement is supposed to be funny in its irony - I apologise). I genuinely identfy with a lot of these self-inflating and other-deprecating tendencies, and this was very enlightening as I will now have in mind to identify them and the extent to which they are flawed.

I believe that the 3Blue1Brown video is a bad example for the point you are trying to make. 
His videos are made for an audience who already have some kind of background in abstract mathematics, so his communication is in-fact comparatively approachable for its target audience.

Overall I agree with this article but found this example quite ironic. 

(The post was not written by Eliezer, but by chaosmage).

I love this table for it's relatability. However, I felt lost in the text putting the table into context. Specifically the sentence,

"But when the rest of us talk about rationality we still mostly talk in abstracts.".

What is really actually meant by this? Both, what is meant by "rationality" in general and what examples should be thought of for "abstractions", are lost on me. Does it apply to everyday talk with friends, what are abstractions there? Does it apply to truth seeking communities like this forum, and what abstractions are used here? Does it appply to scientific communication, other than when using formulas instead of text to carry the concept to the audience?

I feel like what it means is - even when you think you're speaking rationally & logically - your words may still be misunderstood. Message may be ambiguous and perceived incorrectly by the receiver. As communication is so complex, it stresses our need to be conscious of our message, to be concrete (with evidence and examples if applicable), clarify instead of making assumptions and to incorporate active listening (tone, body language, content holistically) when listening. 

There's a typo: 

They are failing to communicate their understanding, or to give unambiguous evidence they even have it.


 IF they even*

Curated and popular this week
Relevant opportunities