Anthony DiGiovanni

2 months ago | lesswrong.com | Anthony DiGiovanni

(This post draws heavily on earlier writing co-authored with Jesse Clifton, but he’s not listed as an author since he hasn’t reviewed this version in detail.)Should we always be able to say whether one outcome is more likely, less likely, or exactly as likely as another? Or should we sometimes suspend judgment and say “none of the above”, that the answer is indeterminate?

Nov 5, 2024 | lesswrong.com | Anthony DiGiovanni |Jesse Clifton

In our jobs as AI safety researchers, we think a lot about what it means to have reasonable beliefs and to make good decisions. This matters because we want to understand how powerful AI systems might behave. It also matters because we ourselves need to know how to make good decisions in light of tremendous uncertainty about how to shape the long-term future.

Jul 31, 2024 | lesswrong.com | Anthony DiGiovanni

An alternative to always having a precise distribution over outcomes is imprecise probabilities: You represent your beliefs with a set of distributions you find plausible. And if you have imprecise probabilities, expected value maximization isn't well-defined. One natural generalization of EV maximization to the imprecise case is maximality: You prefer A to B iff EV_p(A) > EV_p(B) with respect to every distribution p in your set.

Jul 17, 2024 | lesswrong.com | Nicolas Macé |Anthony DiGiovanni |Jesse Clifton

Agents might fail to peacefully trade in high-stakes negotiations. Such bargaining failures can have catastrophic consequences, including great power conflicts, and AI flash wars. This post is a distillation of DiGiovanni et al. (2024) (DCM), whose central result is that agents that are sufficiently transparent to each other have individual incentives to avoid catastrophic bargaining failures.

Mar 5, 2024 | lesswrong.com | Anthony DiGiovanni

Suppose you’re reflecting on your views on two thorny topics: decision theory and anthropics. Considering decision problems that don’t involve anthropics (i.e., don’t involve inferences about the world from indexical information), you might find yourself very sympathetic to evidential decision theory (EDT).