Thank you for the thorough reply!

I'm unsure what "the way it is done" refers to here: is it the general approach loosely illustrated in the first couple of figures, which covers a very broad range of possible graded accounts, or the specific (not firmly endorsed) example indices D or CD?

Sorry, this was pretty vague. I refer to an even larger category: a rule that takes the representor as an input and then wants the output to say something about the comparative justification/arbitrariness/etc. of belief. Most of my comments below are ultimately about reiterating this point.

I think it's worth questioning the claim that maximality is the only or most plausible decision rule in this context.

I agree it's worth questioning. However, with the explicit setup in my previous comment (representor via incompleteness+wanting to get action guidance from some method that uses a probability function) maximality seems undeniable. I think any changes to maximality being correct, or the only correct rule, come from changing or adding something to the setup. 

Why assume a sharply binary picture?

The situation is that I'm trying to figure out which probability functions an idealized version of me could have. Here it seems clear that anything less plausible than something else available is not worth considering. (Even if we did want to consider these, the narrower representor should still be considered first. Having more functions in the representor can only remove strict preferences, so the secondary consideration done with the larger representor never gives action guidance.) If some function is more plausible than another, that other one should not be considered. Having multiple probability functions comes from incompleteness in the plausibility relation, and incompleteness is not graded. I don't see this construction having a sense in which we would draw the boundary of the representor.

Just incompleteness would settle every function in or out, so would not allow for vague endpoints. I attribute vague endpoints to indeterminacy in that it can be unsettled whether one function beats another. That is still not a ranking by degree, which is what the graded picture would need.

On G&S (1982), C&F (2009), Hill (2013, 2019). I’m familiar with Hill (2013) and I skimmed and read summaries of the other literature you mention here. Please correct me if I’m wrong, but they all seem to assume a structure beyond a representor and build a decision rule on that without justifying the assumed structure.

This is relevant for outliers in the following sense: extremal outlier distributions are those that are most vulnerable to alternative precisifications, and thus their inclusion or exclusion can depend on arbitrary precisifications (if we insist on the binary picture).

By "extremal outlier distribution" I take it you mean something about how well supported a function is by our reasons. I don't have an idea what property of a probability function could track that (I'm genuinely interested in finding such properties but pessimistic). In the structural sense extremal means not being a mixture of two other functions in the representor, and that doesn't seem to relate to plausibility. Also, which structurally extremal functions produce the endpoints/midpoint of an EV interval is specific to the option, so I don't see a way to identify endpoint/midpoint functions.

The torture example, and to my reading the whole argument, seems to work on an intuition about the size or total authority of different parts of the representor compared to others. Even in the “one” vs “all the rest” case it is not clear to me how to weigh these up in a non ad hoc way. Justifying that would require finding a size or total authority measure on the representor. (For clarity, I’m not saying it needs to be very formal.) The construction of the representor by incompleteness seems to deny the existence of such a measure. 

If the plausibility relation is graded in some way and the representor is a coarse version of it, continuity is definitely desirable. On the incompleteness (and possible indeterminacy) view, a rule that doesn't flip the verdict seems to misrepresent the situation. As stated before, I don’t think incompleteness construction has a sense in which we would draw a boundary.

My takeaway is that the fundamental disagreement is about where the representor comes from. I’m quite convinced that if we have a representor, it is due to incompleteness. What would produce a graded one?

Thanks for the reply! 

The memory example is interesting. I now think my rejection of (*) (from my previous comment) was too strong but I haven’t figured out what to make of this more precisely. I feel like there is something valuable to be said about different ways that options can exist or be available and how relevant they are to maximality or other decision rules. I’ve always thought of menus as very unambiguous objects, so this is difficult to wrap my head around. I have more specific thoughts about the example below.

I share your intuition for B being impermissible. Having witnessed B being dominated makes it very hard to say that it would be permissible. I am, however, skeptical of my intuition because it needs to make assumptions that I’m not convinced of.

The situation has two conditions: (1) I’m sure that if I remembered what C was, it would be a valid option at the current situation and that it dominates B. Being sure in this way without knowing what C is requires the situation to be the same in every way that could affect these. (2) I’m unable to generate the same options as the other time. I think this can happen in three ways: (i) the inputs to that generation are different, (ii) the generation process is different or (iii) the generation process is just nondeterministic (or possible to run incompletely). Both (i) and (ii) make fully trusting the similarity of the situation implausible so (iii) must be true to satisfy the conditions. As far as I can tell, for the intuition of deferring to a different run of the process to make sense, these assumptions need to be made:

• A common reference menu (not necessarily objective/true) must exist, and the process must sample it.
• Knowledge of dominance relations on the reference menu is authoritative over relations on samples of it.

I notice that if I consciously deny these, I don’t get the intuitive verdict. 

Another example to test the second assumption in isolation: An undeniably trustworthy oracle tells me that an option exists that would dominate B but not A^[1]. My intuitive verdict again is that it would be pretty incredible to take B. I feel this intuition is somewhat weaker than in the memory example but still clear. 

I read “an option exists” as it being in my menu (because that is at least intuitively what being an option means) and then give it similar privileges as real options. Thinking about it more critically, I have a hard time accepting objects that I cannot choose to play in my decision rule for my objects of choice. In terms of the assumptions, I take the oracle to have access to relations on the reference menu but I’m still unsure if I’m willing to let those be authoritative. 

1. ^{^}

    Or similarly: It is a logical implication of my information. This is the case with mixed options over nowhere-optimal options vs somewhere-uniquely-optimal options. 

I’m glad to see more discussion on what to do if maximality does not give action guidance. I think identifying degrees of justification is very compelling. However, I’m sceptical of this being possible to do the way it is done in this post.^[1] 

I understood this post as working on the assumption that there is something meaningful to be extracted from the representor besides unanimity. In such a case, maximality indeed would fail to use that information and I think that any plausible permissibility rule would need to use it. Below is my reasoning for not granting that assumption.

1. We have an intuition about wanting to maximize welfare impartially. 
2. We have made that intuition precise with a method (call it (*)) that involves computing the expected value of a measure of welfare in the cosmos using a probability function over our algebra of propositions. 
3. We carry limited information to constrain that probability function so there are multiple ways to make it as the method needs. 
   1. Each probability function carries no information beyond being one admissible way to represent the information we carry.  
   2. This construction does not imply any sense in which another probability function made this way is anything (e.g. outlier, better, less arbitrary, more grounded, more average, ...) compared to another or that the amount of these functions, or amount of functions with some property, matter. 
4. Therefore, if any probability functions disagree on the comparison of the (*)-method EV of two objects of choice, that tells us that the information used to make the probability functions does not constrain them in a way that would lead to either option being better. 
5. Thus, maximality over the representor is the correct account of our reasons determining if one object of choice is better than another according to method (*).

Notice that this is not about determining if those options are permissible in general, just permissible according to (*). This does not claim to be an “exhaustive criterion of comparative justification” as you say. There can be other ways to make the intuition, or weakenings of it, precise and you can use other normative reasons for choice. For example, my intuition behind the torture example can easily be explained by things outside (*). You don’t need to be clueful wrt. impartial altruism to know not to kick puppies. For instance, see MNB or LF.

What is the meaningful thing you want to extract from the representor? What information does magnitude or being an outlier carry?

1. ^{^}

   I personally tried and failed to do something related with -filtering which you might find interesting. 

This was interesting and had me thinking for a while. This made me consider mixed options as a very relevant consideration for clueless agents. I have some more specific thoughts below. I consider 1. and 5. to be somewhat serious objections to your argument but 2.-4. not so much.

1. 

It seems hard to deny that mixed options are possible, and it directly follows from maximality that when they dominate some options, those should be impermissible. I however don’t think I agree with the actual substantive claim being made regarding those:

(*) Even if we don’t have one of the mixed options that would dominate the nowhere-optimal option in mind, we should still consider it impermissible. (In fact, the argument against P1 requires that we don’t have these mixtures in mind because otherwise P1 would just agree.)

Assume menu  and  in  where  is the option that looks for mixed options to expand the menu to  and then choose again from  Let  be a nowhere-optimal option. If we don’t expect to find a mixed option that dominates , it does not seem impermissible to me either. If we do expect to find one, then that should be reflected in  dominating  which would rule  impermissible. That is to say that I don’t think the nowhere-optimal option is impermissible by virtue of being nowhere-optimal but by the dominating option actually being available. (Or to put it another way: the convexification of the menu used to rule nowhere-optimal options impermissible needs to be earned instead of just stipulated)

2.

To my understanding, your argument of choosing somewhere-uniquely-optimal options is about them being securely permissible (by (a) being undominated and (b) being secure from being nowhere optimal and therefore being ruled out by (*)) and not about other options being impermissible in relation to them. That is to say that you are not arguing for E-admissibility over maximality but merely showing that somewhere-uniquely-optimal options are safe. I read the text as being about how to avoid doing things you shouldn’t rather than giving guidance what you should do. 

3. 

There is something decision theoretically fishy about mixed (by lottery) options:

• As Jesse Clifton pointed out, going along with these involves a dynamic inconsistency. Keeping the preference to carry them out always needs some form of commitment to the plan. Incomparability itself needs it sometimes but needing it always seems noticeably worse.
• Randomizing what we might do seems to strictly lose us information (our choices being part of the world). Having willingness to pay to mix options then violates the “no negative VOI” -intuition.
• These might not be such a large issue since mixed (by lottery) options are never somewhere-uniquely-optimal so don’t need to be chosen anyway, assuming of course that the mixed options are also in the menu. (This is slightly strange on its own too.) The preference for these does still exist. (I noticed this thanks to Michael St. Jule’s comment)

4. 

Nowhere-optimality becomes less common, and somewhere-uniquely-optimality becomes more common the more variation the representor has. That is to say that the decision rule gives us less determinate action guidance, the more clueless we are. This makes sense but still feels a bit awkward for a decision rule made to help with cluelessness to do. This combined with what Clifton noted about it likely being very difficult to find somewhere-uniquely-optimal options, makes the decision rule seem not very action guiding (in that there are many options we could choose) but also quite difficult to follow (in that they are hard to find). I think Jim Buhler’s comment is related to this too.

5. 

Even granting (*) and the decision rule following it, I think the conclusion of ending up almost where we started is too strong even with the softening by “almost”. The behavior is similar, but the idea is extremely different. I read where we started (i) as maximizing EV relative to the (subjective) probability distribution and where we end up (ii) as maximizing EV relative to some permissible (subjective) probability distribution. The preciser doing (i) has a sense in which they are required to choose an option because it is the best or one of the equally best options and they are required to use the same p for every decision. The impreciser doing (ii) is merely doing it because they have to choose something and avoid choosing impermissible options. The impreciser can use one probability function for a decision and another one for the next. Doing (i) allows thinking that you have chosen the best option by your reasons and doing (ii) allows thinking that you didn’t choose against your reasons.

I’m glad to see a more thorough inspection and defense of this version of bracketing. I accepted the rejection given in the paper and had not thought about it since. This post made me consider BUB as a live option to resolve cluelessness for those sympathetic to a person-centred view of consequentialism (though I am personally not). Below are some more specific thoughts. I wouldn’t consider any of them as very serious objections.

• I think it is important to state just how bad it is that BUB can permit options that non-bracketed consequentialism forbids. We can know that on balance A is not permissible, but BUB still picks A (this is the case in the first example). We turn to bracketing because we want action guidance in situations where our candidates for idealized selves^[1] conflict, but then BUB chooses something that we know none of them would pick. This makes me consider BUB as being quite far from my idea of consequentialism. I think you expect this response and that this is what you would, fairly, call a narrow view of consequentialism.
• I think most ways of thinking about altruism are person-based, but impartial welfarist consequentialism specifically is not. To me, what matters is the aggregation, not the individuals. My sympathy for impartial welfarist consequentialism is due to intuitions about the value of experiences as opposed to those more directly about helping people.
• This statement might be too strong: “Persons’ complete prospects provide a non-arbitrary, morally motivated decomposition of consequences”. The discussion in other comments about non-identity is a good example. It seems that more far-out considerations (e.g. acausality) can especially break individuations and it would be good if decision rules made for cluelessness did not at least immediately break from those (not that space-time locations would work here either).
• For me, I do not consider the infinite shift case valid to draw intuitions from even though I agree that we should prefer a world where everyone has x more welfare. I take welfare per-moment to be bounded, so the way to get arbitrary levels of welfare for a person is for that person to have an arbitrarily long life. However, I feel like my idea of a person cannot accommodate having experiences that are arbitrarily far apart. FWIW, having experience moments as the units would also respect the intuitive verdict.
• In the firing line example, it seems like you restated the person-centered view rather than argued against Wilkinson’s spacetime argument. You are effectively moving these people ({p2, p4, p6, p8, p12, ...} and {p5, p15, p25, p35, p45, ...}) into new lines and then saying to Wilkinson that those lines are equally dense with people getting killed.

(This might not be very relevant to the conversation, but I think it is still an interesting point of comparison between BUB and TDB.) Both BUB and TDB can cycle and need some form of committing to plans to avoid getting pumped. The paper uses wise choice for this. In addition to adopting the general rule, using it requires assigning an acyclic relation to determine which plans are feasible for the actual ranking to choose from. In the paper, this is done for TDB in a way that I would consider merely a weak form of commitment. It requires committing to not deviate from the plan at nodes, but it is weak in that the commitment comes from already defined more important reasons constraining less important ones at the nodes. The more important reasons are the rankings non-bracketed consequentialism gives (), and the less important ones are what TDB gives. 

Denying that bracketing must extend non-bracketed consequentialism means that BUB cannot consider  to be more important reasons or even reasons on the same level as its own. We should therefore look for another relation. For TDB,  is just TDB without the bracketing, and an analogue exists for BUB: Let , as the notation suggests, be ᴮᵁ restricted to . This is is a subrelation of both  and ᴮᵁ. Conceptually this is about comparing aggregated welfare in situations where every person is determinate, which is also what the intuition behind BUB would like to do if it wasn’t clueless. (Another possibility is a relation defined on personwise pareto but that does not aggregate at all, and it can fail to block some plans that BUB disprefers in aggregate even if they are not pumps.)

I’d say that this is somewhat less principled as an authority and constitutes less important reasons than  for TDB since is defined before entering the discussion on bracketing whereas  is derived from BUB itself. This makes me think that BUB requires a slightly stronger form of commitment than TDB. 

1. ^{^}

   If we take, as usual, the representor model to contain every way our idealized self might assign credences.

Thanks, these are good points to clarify.

First: 

Why is the move to a lower-ranked value like beauty more legitimate than moving to a utilitarianism conditional on ex post neartermism?

I don't think empirical neartermism exists separately in a way that you could move onto it from non-neartermism. Doing that requires carving up the empirical space into an ordering which I don't know how to do (see this open question: How could the intuition “Less arbitrary parts of beliefs could be lexically more important than more arbitrary parts and that could be used for filtering” be made precise and action guiding when arbitrariness is read as being about what the beliefs are based on?). That is also what we are denying to be able to do when we work with an unordered representor.
 
Taking ex post neartermism as an empirical stance, it lives in the representor. I'd read having an overall (as opposed to being in one probability distribution in the representor) 10% credence in it as: For any options A and B and for any probability distribution in the representor, propositions that are over, say, 100 years away are the same with 10% chance. That is to say that if you believe 10% in ex post neartermism, that is just what utilitarianism is for you. There is no separate empirically non-neartermist utilitarianism to consider before neartermist utilitarianism.

In contrast, it seems like you are interested in considering those as separate probability distributions in order: take p_non-near and p_near. Then you could do the filtering indexed on those two and get the action guidance from being clueful in the near term. However, this would imply having lexical order between those which is not compatible with having cardinal credence between them: having 10% credence in p_near would imply collapsing those into p_mix=0.1*p_near+0.9*p_non-near and we get the thing I said in the previous paragraph.

All this is to say that these three are incompatible: (1) the difference is empirical, (2) they are separate (in the sense that you could condition on the other and be clueful) and (3) you have cardinal credence between them. Just having (1) and (2) together is possible but that requires the structure in the empirical space to separate them. Hill's confidence ranking stuff (see footnote 14) and the open question I mentioned are related to this.

On normative views being privileged units: Do you mean as privileged over carving up the empirical space into units? If so I'd read that point as "Why use normative views as the units instead of empirical units?". Again, if we do have a unique way to carve up the empirical space in a meaningful way, I think that is worth doing. Also, the parity between carving the normative space and carving the empirical space doesn't bite for LF like it might for MNB: Taking the parts of the normative space that have lexicality between them isn't really carving it into units. Lexicality already carves them and we just use that. If there is no lexicality, we don't have the units.

Second: 

I'd think of beauty as a part of a rank that orders options based on just general vibes-based aesthetic stuff. Everything with what I'd consider to be "harder" normative content is above that rank (like welfare, deontology, virtue and even some vibes-based self interest). I also don't really have anything that would come after the rank involving beauty.

(I’m posting this as a separate comment for clarity)

I think it is important to state just how different CHA is from impartial consequentialism. I read your text as claiming that this is obviously what impartial consequentialism is under constraint (for example based on your “There is no option to avoid making choices, so complaining about having to do it imperfectly would go nowhere.” and “we are not so much given a possibility to be (kind of) rational in our choices as forced to apply the limited degree of rationality that's available.”) The problem is if (1) that's still the thing we cared about, (2) if it actually works and (3) if we are actually forced into it. 

1. Limiting impartial consequentialism to a subset is fundamentally different from true impartial consequentialism, not just an imperfect way of doing it. Going from “Increasing welfare on balance” to “increasing welfare in a proper subset while knowing that the effect is indeterminate on balance” is a huge leap to me. I’m sympathetic to the latter one too but that requires independent motivations (like virtue). The whole deal with impartial consequentialism is doing good on balance so losing that is significant.
2. I don’t think it works as cleanly as you state it. See my other comment on bracketing.
3. There definitely are places to go besides nowhere (that is to say that we are not forced into CHA to get action guidance): other normative views and limited versions of impartial consequentialism motivated by reasons besides “nowhere else to go”. See TDB (very similar concept to CHA but in the overview Clifton is clear that it is a distinct theory from consequentialism as opposed to just obviously what clueless consequentialists are supposed to do), MNB, LF and possibly some other entries in the competition. 

Option 3 has been made precise with bracketing: overview and paper. I think this shows why it is not clear how "some subset" is defined even when we have a clear idea of what we want it to track (for top-down bracketing: the effects we're not clueless about taken as widely as possible).

The problem with carving up effects is with choosing which locations of value to discount when they have determinate sign on their own (so not clueless) but not in aggregate (so clueless). Clifton gives a clear example in the overview post and this seems to apply to your CHA: an intervention reducing animal product consumption has positive EV for farmed animals, negative for wild animals, and indeterminate in aggregate. I do not think that the test you propose resolves this conflict since both groupings pass it (each has determinate sign on its own). So there may be multiple conflicting ways to draw the cluelessness horizon and picking some privileged way to draw them requires justifying it over all the others. (Note on footnote 11: I don’t think this is waivable with anit-scepticism since these are genuinely different ways to draw the line and not just our inability to draw.) 

Thanks for this post Aaron! I especially value the part about asymptotic structure instead of the sharp thresholds. 

I think it’s useful for those interested in the topic to highlight the connection of your post to existing EA discussion and academic literature on ethics and decision theory. Here are some of the things I’m aware of:

• As @Robi Rahman🔸  points out, the post is arguing for an established view: lexical (negative) utilitarianism. I think The Center on Reducing Suffering’s critique of Toby Ord's blog post “Why I’m not a negative utilitarian” is a relevant discussion of negative utilitarianism that also touches on lexicality and can work as a “common misconceptions to avoid” for people new to the idea.
• Like you, Academian takes a critical look at the vNM axioms and argues against continuity being required in a LessWrong post. They highlight the original lexicality paper: Melvin Hausner’s 1954 “Multidimensional utilities” that weakens the continuity axiom of vNM, producing lexicality.
• Teo Ajantaival has an entire chapter in his book/sequence “Minimalist Axiologies” on “Doesn't this endorse destroying the world?”.
• On a more general note, vNM hasn’t been the favored framework in normative decision theory for a while. The frameworks of Savage and then Jeffrey-Bolker have provided increasingly realistic/reasonable setups while keeping the utilities and most of the properties of the axioms. I think Richard Bradley’s 2017 book “Decision Theory with a Human Face” provides the canonical background for current normative decision theory.
• In contrast to what you argue in footnote 21, completeness and transitivity have both been challenged (I think successfully). Especially Suzumura consistency and representor models (more on incompleteness in belief in Anthony DiGiovanni’s comment) are attractive alternatives as they are more appropriate for real agents like us and still invulnerable to value pumps.

Also, I think that in a precise baysian framework, the strongest argument against lexicality is that it is irrelevant because the EV of two options will almost never be exactly the same. This changes if we incorporate imprecision because then it is quite possible that the primary utility doesn’t provide a preference over two options and instead gives comparative indeterminacy which one could consider similar to indifference for the purpose of lexicality. That is to say, I think lexicality is more action-guidance relevant and conceptually attractive in an, arguably better, imprecise framework.