User:Dv82matt/sandbox

Newcomb’s problem is the subject of entrenched confusion here on Less Wrong. I will demonstrate that contrary to the consensus here one boxing does not generally win on Newcomb like problems.

First I will point out that no one, so far as I am aware, has actually made a credible argument that one boxing wins. Usually it is just assumed as in Eliezer’s original post on the subject (Newcomb's Problem and Regret of Rationality) where the assumption that one boxing wins is the first step in a chain of reasoning that concludes that one boxing is rational. I can only presume that if Eliezer had started with the assumption that two boxing wins he would have concluded that two boxing is rational.

Other times the assumption is pushed back a step and it is assumed that the predictor is sufficiently smart that the prediction and your decision whether to take one box or two are both causally connected to the initial conditions. But that assumption is not included in the statement of Newcomb’s problem so its introduction as a prior is unjustified.

Here at Less Wrong the dominant hypothesis for how the prediction is made is probably the simulation hypothesis. But that is merely one possibility among many. Depending on how the prediction is made, two boxing absolutely can win. If you doubt this consider the simple example of an agent that always predicts two boxing. It doesn’t scan minds or simulate your decision algorithm or engage in any complicated computation whatsoever. It is a dumb predictor. It just predicts two boxing. Two boxing absolutely wins in this case. You might object to the idea that such an agent could be so highly accurate but it may not be as improbable as the hypothesis that the dilemma is being proposed by a being that has simulated you with near perfect precision.

So where does the idea that two boxing wins come from? Even those who disagree that one boxing is rational seem to concede that one boxing wins. Why? I think it is likely due to unstated assumptions. Assumptions that are not intended to factor into the dilemma yet which are smuggled in nonetheless and affect our perception of what strategy wins.

Here on Less Wrong the predictor is usually a super intelligence referred to as Omega. This is problematic for a few reasons. First, it tends to give a the simulation hypothesis (and other computationally expensive hypotheses) undue weight. It is difficult to credit that a being that has vast computational resources and other futuristic technology readily available might not make use of it to generate the highly accurate and seemingly difficult prediction. But Omega is supposed to be an idealizing assumption not an excuse to smuggle in assumptions about the level of tech used to make the prediction.

Second, it invites the faulty simplification of declaring Omega infallible. The rationalization is that a being that is correct some arbitrarily high percentage of the time is nearly the same as a being that is infallible so why not just say Omega is infallible. It is easier to just specify that Omega is a perfect predictor. But it doesn’t work. In the context of Newcomb’s problem a being that is correct 99.999% of the time is not at all akin to a being that is infallible.

To make the point clearer lets compare a fallible and infallible agent. We will call the infallible agent Alpha, but in order to make a comparison with the fallible agent possible we will specify that Alpha deliberately makes the wrong prediction roughly 40% of the time. So Alpha makes the correct prediction about 60% of the time. Should you one box when Alpha presents you with Newcomb’s problem? Of course you should. The initial conditions that are causally connected to the choice you will make are also causally connected to Alpha’s prediction with 60% certainty which still results in one boxing having greater expected utility than two boxing.

Now consider Beta. Beta is not infallible but like Alpha makes the correct prediction about 60% of the time. Should you one box in this case? No. Of course not. The initial conditions are highly unlikely to be causally connected to Beta’s prediction. In this case two boxing is much more likely to have greater expected utility than one boxing.

Making Omega infallible rebalances the problem in favor of one boxing. I used a low percentage (60%) in the example to make the difference between fallible and infallible agents more obvious, but I don’t know of a reason to believe that the bias becomes less severe as the accuracy is increased.

Idealizing Newcomb’s problem properly is difficult. Adding Omega to the problem was an attempt to do so. It is a failed attempt but one can understand the motivation behind it. Assuming the predictor is human introduces a set of unstated assumptions and Omega was an attempt to remove them. If the predictor is human then the predictor may be unlikely to be sufficiently smart to qualify as a smart predictor. There are also assumptions about the motivations of humans. For example it is difficult to credit that a human would part with a million dollars as easily as a thousand so we might assume a human bias toward predicting two boxing. Omega does address these smuggled assumptions but introduces several of its own. Omega short circuits Occam’s Razor. It gives undue weight to hypotheses that require vast complexity. Remember Omega is supposed to be an idealizing assumption not an excuse to smuggle in assumptions about the complexity of the method of making the prediction.

Newcomb’s problem appears to provide us with sufficient information to allow us to definitely conclude which strategy wins, but actually it doesn’t. It is an illusion of sufficient information. There are basically two versions of Newcomb’s problem that tend to be prominent the smart predictor version and the dumb predictor version. Assumptions get smuggled in in favor of the preferred viewpoint and it becomes reinforced. Once someone has settled on one version or the other they will discover that they now have sufficient information to arrive at a definite conclusion.

I’m sure most of the one boxers here at Less Wrong have considered and dismissed the argument that since the million is either there or it isn’t and further that since two boxing wins in both cases that two boxing must win overall. Under the dumb (or insufficiently smart) predictor hypothesis this argument is absolute and its dismissal reeks of illogic. Under the smart predictor hypothesis however this dismissal makes sense. After all whether you get the million is causally connected to the initial conditions that also determine the choice you actually make. So committing to one boxing no matter what makes sense.

The problem here is that Newcomb’s problem doesn’t actually state whether you are dealing with a smart predictor or a dumb predictor. It doesn’t state whether Omega is sufficiently smart. It doesn’t state whether the initial conditions that are causally connected to your choice are also causally connected to the prediction Omega makes. So without smuggled in assumptions there is insufficient information to determine whether to one box or two box. You might as well flip a coin.

Further thoughts:

• Since one boxers cannot gain by successfully deceiving Omega but two boxers can, it is plausible that for a given level of complexity two boxing predictions are more accurate than one boxing predictions. This asymmetry would allow for a selection effect. If Omega only ever offered the choice to those it predicted would two box it may lower the complexity requirements of Omega significantly. This would be a case where the two boxers always get $1,000 and the one boxers very rarely receive the offer but when they do it means that Omega made a mistake and so they get nothing.

• Probabilities often seem to be misunderstood or ambiguous in Newcomb’s problem. If we say that Omega is 99% likely to be correct is that a statement about reality or is that a statement about a map of reality? Outside of quantum mechanics probabilities are statements about the map not the territory. They represent how sure or unsure we are of the map. In the context of Newcomb’s problem probabilities often seem to be taken as statements about the territory. This has the same effect as making Omega into an infallible agent that deliberately predicts wrongly some tiny percentage of the time.

• Swapping the predictor and chooser roles may serve as an intuition pump. Instead of imagining that Omega is presenting you with Newcomb’s problem imagine that you are presenting Omega with Newcomb’s problem. In attempting to predict whether Omega will take one box or two, how do you ensure that your prediction is maximally accurate?

• A factor that tends to be overlooked in Newcomb’s problem is that reporting the high likelyhood of the accuracy of the prediction to the chooser actually confounds the ability to make an accurate prediction over and above the mere difficulty of making an accurate prediction in the absence of such reporting.