Dmitry Vaintrob

Featured in: Favicon

arxiv.org

Articles

Renormalization Redux: QFT Techniques for AI Interpretability — LessWrong

Jan 17, 2025 | lesswrong.com | Lauren Greenspan |Dmitry Vaintrob

In a previous post, Lauren offered a take on why a physics way of thinking is so successful at understanding AI systems. In this post, we look in more detail at the potential of Quantum field theory (QFT) to be expanded into a more comprehensive framework for this purpose. Interest in this area has been steadily increasing, but efforts have yet to condense into a larger-scale, coordinated effort.
The quantum red pill or: They lied to you, we live in the (density) matrix — LessWrong

Jan 17, 2025 | lesswrong.com | Dmitry Vaintrob

Today's post is in response to the post "Quantum without complications", which I think is a pretty good popular distillation of the basics of quantum mechanics. For any such distillation, there will be people who say "but you missed X important thing". The limit of appeasing such people is to turn your popular distillation into a 2000-page textbook (and then someone will still complain).
Permanents: much more than you wanted to know — LessWrong

Jan 16, 2025 | lesswrong.com | Dmitry Vaintrob |Alexander Gietelink Oldenziel

Today's "nanowrimo" post is a fun longform introduction to permanents and their properties, organized in the way I wish it had been explained to me. Note before going on: this is not my field, and this post is even more likely than usual to contain incorrect statements or bugs. Recall that given an n×n matrix A with coefficients Aij, its determinant is a sum over permutations σ on n letters with j=σ(i), det(A)=∑σ(−1)σ∏iAiσ(i). Here the notation (−1)σ denotes the parity of a permutation.
Basics of Bayesian learning — LessWrong

Jan 14, 2025 | lesswrong.com | Dmitry Vaintrob

See also: the “preliminaries” section in this SLT intro doc. This is a preliminary post for the series on “distilling PDLT without physics”, which we are working on joint with Lauren Greenspan. The first post in this series is my post on the “Laws of large numbers” (another preliminary) which is completely independent of this one.
Paper club: He et al. on modular arithmetic (part I) — LessWrong

Jan 13, 2025 | lesswrong.com | Dmitry Vaintrob

In this post we’ll be looking at the recent paper “Learning to grok: Emergence of in-context learning and skill composition in modular arithmetic tasks” by He et al. This post is partially a sequel to my earlier post on grammars and subgrammars, though it can be read independently. There will be a more technical part II. I really like this paper. I tend to be pretty picky about papers, and find something to complain about in most of them (this will probably come up in future).
The purposeful drunkard — LessWrong

Jan 12, 2025 | lesswrong.com | Dmitry Vaintrob

I'm behind on a couple of posts I've been planning, but am trying to post something every day if possible. So today I'll post a cached fun piece on overinterpreting a random data phenomenon that's tricked me before. Recall that a random walk or a "drunkard's walk" (as in the title) is a sequence of vectors x1,x2,… in some Rn such that each xk is obtained from xk−1 by adding noise.
Dmitry's Koan - LessWrong

Jan 9, 2025 | lesswrong.com | Dmitry Vaintrob

I can be a nightmare conference attendee: I tend to ask nitpicky questions and apply a dose of skepticism to a speaker's claims which is healthy in doing one's own research, but probably not optimal when everyone else is trying to follow a talk. I'm working on being better at this, but for now I blame my background. There is one nitpick that comes up again and again. In fact in one conference I brought it up so often that Jake Mendel coined a term for it: "Dmitry's koan".
Deep Learning is cheap Solomonoff induction? — LessWrong

Dec 7, 2024 | lesswrong.com | Lucius Bushnaq |Dmitry Vaintrob

TL:DR: Recently, Lucius held a presentation on the nature of deep learning and why it generalises to new data. Kaarel, Dmitry and Lucius talked about the slides for that presentation in a group chat. The conversation quickly became a broader discussion on the nature of intelligence and how much we do or don't know about it. Lucius: I recently held a small talk presenting an idea for how and why deep learning generalises.
Toward Safety Case Inspired Basic Research — LessWrong

Oct 31, 2024 | lesswrong.com | Lucas Teixeira |Lauren Greenspan |Dmitry Vaintrob |Eric Winsor

AI safety is a young science. In its early history, deep and speculative questions regarding the risks of artificial superintelligence attracted the attention of basic science and philosophy, but the near-horizon focus of industry and governance, coupled with shorter safety timelines, means that applied science is now taking a bigger hand in the field’s future.
Mapping the semantic void: Strange goings-on in GPT embedding spaces — LessWrong

Dec 15, 2023 | lesswrong.com | Dmitry Vaintrob |Joseph Miller |Oliver Sourbut |Carl Feynman

TL;DR: GPT-J token embeddings inhabit a zone in their 4096-dimensional embedding space formed by the intersection of two hyperspherical shells. This is described, and then the remaining expanse of the embedding space is explored by using simple prompts to elicit definitions for non-token custom embedding vectors (so-called "nokens").