Joseph Van Name

Featured in:

Articles

Deep Forgetting & Unlearning for Safely-Scoped LLMs — LessWrong

Dec 5, 2023 | lesswrong.com | Thomas Kwa |Bogdan Ionut Cirstea |Roman Leventov |Joseph Van Name

Thanks to Phillip Christoffersen, Adam Gleave, Anjali Gopal, Soroush Pour, and Fabien Roger for useful discussions and feedback. This post overviews a research agenda for avoiding unwanted latent capabilities in LLMs. It argues that "deep" forgetting and unlearning may be important, tractable, and neglected for AI safety. I discuss five things. The practical problems posed when undesired latent capabilities resurface.
Provably Safe AI — LessWrong

Oct 5, 2023 | lesswrong.com | Peter McCluskey |Joseph Van Name |Start AtTheEnd

I've been hearing vague claims that automated theorem provers are ableto, or will soon be able to prove things about complex software such asAIs.Max Tegmark and Steve Omohundro have now published a paper, ProvablySafe Systems : The Only Path To ControllableAGI, which convinces me that this isa plausible strategy to help with AI safety. The basic steps:Write trustworthy AIs that are capable of searching for proofs andverifying them. Specify safety properties that we want all AIs to obey.
Logical Share Splitting — LessWrong

Sep 11, 2023 | lesswrong.com | Joseph Van Name

Thank you! Much to think about, but later... If there are a large number of true-but-not-publicly-proven statements, does that impose a large computational cost on the market making mechanism? I expect that the computers running this system might have to be fairly beefy, but they're only checking proofs. They're not, though. They're making markets on all the interrelated statements. How do they know when they're done exhausting the standing limit orders and AMM liquidity pools?
My tentative best guess on how EAs and Rationalists sometimes turn crazy — LessWrong

Jun 21, 2023 | lesswrong.com | Ben Pace |Nina Rimsky |Aaron Bergman |Joseph Van Name

Epistemic status: This is a pretty detailed hypothesis that I think overall doesn’t add up to more than 50% of my probability mass on explaining datapoints like FTX, Leverage Research, the LaSota crew etc., but is still my leading guess for what is going on. I might also be really confused about the whole topic.
Work dumber not smarter — LessWrong

Jun 1, 2023 | lesswrong.com | Joseph Van Name |Gesild Muka

The intersection of machine learning and generalizations of Laver tables seems to have a lot of potential, so your question about this is extremely interesting to me. Machine learning models from generalized Laver tables? I have not been able to find any machine learning models that are based on generalized Laver tables that are used for solving problems unrelated to Laver tables.
Why and When Interpretability Work is Dangerous — LessWrong

May 28, 2023 | lesswrong.com | Joseph Van Name |Charlie Steiner |Zheng Wang |Arthur Conmy

Many new researchers are going into AI alignment. For a variety of reasons, they may choose to work for organizations such as Anthropic or OpenAI. Chances are good that a new researcher will be interested in "interpretability". A creeping concern for many: "Is my research going to cause AGI ruin? Am I making the most powerful AI systems more powerful, even though I'm trying to make them safer?" Maybe they've even heard someone say that "mechanistic interpretability is capabilities research".
'Fundamental' vs 'applied' mechanistic interpretability research — LessWrong

May 23, 2023 | lesswrong.com | Lee Sharkey |Logan Riggs |Joseph Van Name |Arthur Conmy

When justifying my mechanistic interpretability research interests to others, I've occasionally found it useful to borrow a distinction from physics and distinguish between 'fundamental' versus 'applied' interpretability research. Fundamental interpretability research is the kind that investigates better ways to think about the structure of the function learned by neural networks. It lets us make new categories of hypotheses about neural networks.