Oliver Sourbut

Jan 16, 2025 | lesswrong.com | Oliver Sourbut

NB this dialogue occurred at the very end of 2023, and for various reasons is only being published ~a year later! Keep this in mind while reading.

Jan 7, 2024 | lesswrong.com | Steven Byrnes |Seth Herd |Oliver Sourbut

Tl;dr: A “deceptively-aligned AI” is different from (and much more specific than) a “deceptive AI”. I think this is well-known and uncontroversial among AI Alignment experts, but I see people getting confused about it sometimes, so this post is a brief explanation of how they differ. You can just look at the diagram below for the upshot. Some motivating context: There have been a number of recent arguments that future AI is very unlikely to be deceptively-aligned.

Dec 27, 2023 | lesswrong.com | David Lorell |Alexander Gietelink Oldenziel |Thane Ruthenis |Oliver Sourbut

Also, what do you mean by mutual information between Xi, given that there are at least 3 of them? You can generalize mutual information to N variables: interaction information. Why would it always be possible to decompose random variables to allow for a natural latent? Well, I suppose I overstated it a bit by saying "always"; you can certainly imagine artificial setups where the mutual information between a bunch of variables is zero.

Dec 16, 2023 | lesswrong.com | Thane Ruthenis |Oliver Sourbut |Mo Putera |Gerald M. Monroe

When discussing AGI Risk, people often talk about it in terms of a war between humanity and an AGI. Comparisons between the amounts of resources at both sides' disposal are brought up and factored in, big impressive nuclear stockpiles are sometimes waved around, etc. I'm pretty sure it's not how that'd look like, on several levels. 1.

Dec 15, 2023 | lesswrong.com | Dmitry Vaintrob |Joseph Miller |Oliver Sourbut |Carl Feynman

TL;DR: GPT-J token embeddings inhabit a zone in their 4096-dimensional embedding space formed by the intersection of two hyperspherical shells. This is described, and then the remaining expanse of the embedding space is explored by using simple prompts to elicit definitions for non-token custom embedding vectors (so-called "nokens").