www.jimbuhler.site
Also on LessWrong and Substack, with different essays.
All opinions are my own.
TL;DR: Numerical weights can very efficiently communicate how you are disposed to adjudicate a wide range of tradeoffs. Stating a threshold reveals your trade-off rates much as betting odds reveal credences.
Agreed! (For reasons we give here and there that echo yours. Great points!) One important remaining question, however, is: are one's precise numerical weights (and implied trade-offs) any less arbitrary than all the equally precise alternative weights?
If yes, great! But I don't think this is quite what you argue for here, right? (Although this is what I was expecting from the title.) You argue for the value of making our judgments explicit, improving transparency and coherence, and making arbitrary decisions harder to justify---one can very well agree with all this while answering "no" to the above question.
If not, then there's a strong case against pulling precise numbers out of thin air (DiGiovanni; Clifton).
From all your adjacent posts and our related comment threads (especially this one), I'm still unsure whether you think we always ought to have precise credences about everything and that these credences are non-arbitrary and hence epistemically uniquely privileged (i.e., reject DiGiovanni's conceptual premise, afaiu). I'm guessing something in between the yes and the no given this interesting comment of yours and everything you wrote here re reasonable disagreements?
I actually think this is a consideration that is very underrated by people trying to reduce long-term risks. Some related references/thoughts here and there. Some people at CLR a few years ago were calling this the "Smokey Bear effect", in reference to campaigns against wildfires that ended up preventing benign wildfires and causing bigger ones (for the same reasons you discuss above, I think?).
I haven't taken the time to engage with your arguments for why pausing AI might overall reduce AI x-risks regardless, but I'm initially skeptical, fwiw. It seems more complicated than what you suggest. (I might go back to this and engage some time in the future.)
Thanks Jelena, great questions! :)
So I’d guess returns sometimes drop in steps rather than smoothly. Is that something your model tries to capture, or is it left to the fund managers’ judgment?
Each fund’s diminishing-returns curve (except GiveWell’s) falls smoothly until it reaches a cap of five times the fund’s current budget, where it drops to zero. So the model has one step, at that cap, but none of the earlier steps you describe. It also assumes each extra $2M can be put to use straight away, so the months spent building systems aren’t in it. Two things soften this. We give to funds that pass the money on to many organisations, and each one hits its bottleneck at a different point, so the fund as a whole loses value more gradually than any single organisation. And fund managers decide which grantees get money and when. That said, our handling of diminishing returns is provisional, and it’s an area we want to improve.
On public funding and GHD: our model currently uses each global health fund’s own cost-effectiveness estimates. At least GiveWell’s already account for what governments and multilateral donors fund. In the future, we definitely would like to check more rigorously that this adjustment is made (and made well) for all our global health funds, rather than relying on the figures the funds give us.
Hey Vasco. I was not involved in making this assumption, so I can only transcribe what my busy colleagues said on this. (This means I very likely won’t be able to reply to follow-up questions, if there are any. But what you or anyone may want to add on this will be read.)
The assumptions come directly from GiveWell itself. It might be true that the marginal cost-effectiveness is lower than their average for 2025, but we don’t immediately see a reason to believe that it equals their bar (which was 8x at time of publishing). We may look into how that’s changed, and where GiveWell’s current cost-effectiveness lies in the near future.
I think we would need to unpack too many things for addressing your first and third paragraphs to be worth it. (And I think you already know what I personally think about the first one.) I'll just note that, tentatively ~bracketing out some effects is not incompatible with valuing further research on them.
many people care not only about increasing welfare in expectation, but also about not decreasing welfare. For those people, the uncertain effects can matter a lot.
I tentatively agree and think this should be the strongest argument against our "indeterminate effects = 0" move, from RP's perspective. (Although I do not understand the most plausible versions of risk/ambiguity aversion well enough to be confident here, and remain sympathetic to this response Laura gave you when you raised the same point elsewhere. Defo still something I'd like to revisit at some point, though!)
From the point of view of decreasing the chance of causing large harm, funding CCF is much worse than doing nothing? This could be an uncomfortable conclusion, but it seems worth communicating if you want to empower donors to act in agreement with their values.
I don't think this part is fair, though. If donating to the CCF is bad because of risk aversion + substantial off-target effects, donating to pretty much anything is bad, then. Again, here, you're raising a potential problem that goes well beyond the CCF. (I understand very well that your views imply prioritizing research, though. And we might come closer to this ourselves if/once we estimate VOI for different projects and account for off-target effects.)
I did mean actually neutral. I assume that our position would be to tentatively assign a value of 0 to effects where any precise number within a range spanning over 0 seems arbitrary, which approximates bracketing out effects we are clueless about. (I know that you are skeptical of bracketing and might say that 0 is just as arbitrary as any other number.)
you could still disclaim you do not know whether the interventions being funded increase or decrease welfare such that other funders update towards funding research decreasing the relevant uncertainties.
I think we overall would disagree with the first part, due to being sympathetic to bracketing out effects we are clueless about, or maybe believing these are not that large to begin with. (I am probably the person on the team most sympathetic to deep uncertainty about this, though.) But fwiw, we are planning on considering including "meta" funds/projects in the allocation, including those that reduce our key uncertainties. (Our key uncertainties may very well differ from yours, though.)
For work on moral weights to inform CCF's allocation, you would have to combine it with research on the effects on non-target individual (for example, estimating the soil-arthropod-years affected per electrically stunned shrimp)?
Well, this depends on what the moral weight research concludes, but in many scenarios, yes, we would need to if we wanted off-target effects to influence the CCF allocation (which, again, we might not want). Fwiw, I don't expect "estimating the soil-arthropod-years affected per electrically stunned shrimp" to be the hardest part, relative to figuring out the sign of this + other potential off-target effects of shrimp stunning (and how to weight all this). And I don't expect the CCF team to figure this out any time soon. The AW team might be more likely to make some progress, but I also don't expect it to figure this out.
Hey Vasco. I wasn't one of the post's authors, but chiming in on RP's general tentative take on modeling off-target effects. (I'll let the others comment on why this was not mentioned in the post, if they want to. But, fwiw, excluding "flow-through effects" is listed as a limitation of our model on the CCF website.)
I agree this is a limitation, and I am tentatively in charge of thinking more about how we could model off-target effects in a non-arbitrary way (which you know I am pessimistic about). This is however low on my/our list of priorities, since I doubt that whatever way of modeling them we might eventually opt for would substantially update the CCF's allocation. We'd most likely remain neutral on whether a given off-target effect is positive or negative, overall, in a way that leaves the precise allocation unchanged.
While this remains a limitation of the model behind the CCF, I think it is a much broader problem that the EA community may want to grapple with. I don't think off-target effects are a reason for anyone to distrust the CCF's current allocation, more than they are a reason to doubt any precise allocation---given the immense imprecision/ambiguity surrounding the scale and (especially) the sign of these off-target effects. And while you know I am myself sympathetic to such doubt, I still see a lot of value in thinking about where money should be allocated, conditionalizing on worlds in which off-target effects and other sources of deep uncertainty are not paralyzing.
I am not aware of any intervention supported by impact-focussed funders which increases animal welfare accounting for effects on soil invertebrates.
RP's work on moral weights and animal welfare might happen to address parts of this concern of yours in the future (and inform our CCF allocation), fwiw. (Macroscopic Ventures' AW team might also do related work.) But you will not find the answers you're looking for from our CCF team on this point.
Two crucial hidden assumptions, which I think you'll agree are worth unveiling and scrutinizing:
First, the "thus" inference does not automatically follow. It may be very hard to actually counterfactually affect all these many future beings. (See #1 and #3 here.) For example, it may be that genuine lock-in scenarios (which I think is what you are most worried about) are very unlikely and/or nearly unavoidable no matter what we do.
Second, assuming the far-future effects of our actions dominate near-term ones, ex post, we should arguably be deeply (if not radically) uncertain about whether these effects help or harm far future beings, which lowers the expected value of trying to.
I think these two assumptions (especially the second one) are the ones that really need justification in a case for longtermism (and I know you've yourself touched on them in previous writings/podcasts). To be clear, there are plausible arguments in their favor, imo. (And I have myself been doing some work on preventing bad space futures for animals.) But there are also plausible arguments against, and I don't think we should be anywhere near as confident as your post suggests that "those serious about animal welfare [...] should be almost entirely concerned with making sure the far future goes well for animals". (Even though I don't think the case for LongtermismxAnimals is any weaker than the more general or human-focused longtermist case, to be clear.)
That said, thanks for discussing this kind of important cause prio considerations which I think should be discussed more. I also like the parts of the posts on what LongtermismxAnimals would imply.