-
2 months ago | 
lesswrong.com | Lucius Bushnaq |Dan Braun |Stefan Hex |Lee Sharkey
This is a linkpost for Apollo Research's new interpretability paper: "Interpretability in Parameter Space: Minimizing Mechanistic Description Length with Attribution-based Parameter Decomposition". We introduce a new method for directly decomposing neural network parameters into mechanistic components. At Apollo, we've spent a lot of time thinking about how the computations of neural networks might be structured, and how those computations might be embedded in networks' parameters.
-
Jan 8, 2025 | 
lesswrong.com | Lucius Bushnaq |Daniel Murfet |Joseph Miller |Matthew Clarke
TL;DR: There may be a fundamental problem with interpretability work that attempts to understand neural networks by decomposing their individual activation spaces in isolation: It seems likely to find features of the activations - features that help explain the statistical structure of activation spaces, rather than features of the model - the features the model’s own computations make use of.
-
Dec 7, 2024 | 
lesswrong.com | Lucius Bushnaq
Note: This is a more fleshed-out version of this post and includes theoretical arguments justifying the empirical findings. If you've read that one, feel free to skip to the proofs.
-
Dec 7, 2024 | 
lesswrong.com | Lucius Bushnaq |Dmitry Vaintrob
TL:DR: Recently, Lucius held a presentation on the nature of deep learning and why it generalises to new data. Kaarel, Dmitry and Lucius talked about the slides for that presentation in a group chat. The conversation quickly became a broader discussion on the nature of intelligence and how much we do or don't know about it. Lucius: I recently held a small talk presenting an idea for how and why deep learning generalises.
-
Oct 26, 2024 | 
lesswrong.com | Lucius Bushnaq |Stefan Heimersheim |Daniel Murfet
Epistemic status: Low confidence. This is a bit of an experiment in lowering my bar for what I make a proper post instead of a shortform. It's just an early stage idea that might go nowhere. It looks to me like neural networks might naturally develop error-correction mechanisms because error correction dramatically increases the volume of solutions in the loss landscape. In other words, neural networks with error correction seem like they’d tend to have much lower learning coefficients.
-
Oct 14, 2024 | 
lesswrong.com | Lucius Bushnaq
Tl;dr: We generalize the mathematical framework for computation in superposition from compressing many boolean logic gates into a neural network, to compressing many small neural networks into a larger neural network. The number of small networks we can fit into the large network depends on the small networks' total parameter count, not their neuron count. Work done at Apollo Research.
-
Sep 24, 2024 | 
lesswrong.com | Eliezer Yudkowsky |Lucius Bushnaq |Thomas Kwa |Joey KL
Crossposted from Twitter with Eliezer's permissionA common claim among e/accs is that, since the solar system is big, Earth will be left alone by superintelligences. A simple rejoinder is that just because Bernard Arnault has $170 billion, does not mean that he'll give you $77.18. Earth subtends only 4.54e-10 = 0.0000000454% of the angular area around the Sun, according to GPT-o1.
-
Aug 8, 2024 | 
lesswrong.com | Lucius Bushnaq |Daniel Murfet
TLDR: In a neural network with d parameters, the (local) learning coefficient λ can be upper and lower bounded by the rank of the network's Hessian d1:d12≤λ≤d12+d−d13. The lower bound is a known result. The upper bound is a claim by me, and this post contains the proof for it. If you find any problems with the proof, do point them out. The learning coefficient λ is a measure of loss basin volume and network complexity. You can think of it sort of like an effective parameter count of the model.
-
Jul 18, 2024 | 
lesswrong.com | Lee Sharkey |Lucius Bushnaq |Dan Braun |Stefan Hex
Why we made this list: The interpretability team at Apollo Research wrapped up a few projects recently. In order to decide what we’d work on next, we generated a lot of different potential projects. Unfortunately, we are computationally bounded agents, so we can't work on every project idea that we were excited about! Previous lists of project ideas (such as Neel’s collation of 200 Concrete Open Problems in Mechanistic Interpretability) have been very useful for people breaking into the field.
-
Jul 6, 2024 | 
lesswrong.com | Lucius Bushnaq
Many people in interpretability currently seem interested in things like enumerative safety, where you describe every part of a neural network to ensure all the parts look safe. Those people often also talk about a fundamental trade-off in interpretability between the completeness and precision of an explanation for a neural network's behavior and its description length. I feel like, at the moment, these sorts of considerations are all premature and beside the point.
