Mikhail Samin

Jan 12, 2025 | lesswrong.com | Mikhail Samin

We have contact details and can send emails to 1500 students and former students who've received hard-cover copies of HPMOR (and possibly Human Compatible and/or The Precipice) because they've won international or Russian olympiads in maths, computer science, physics, biology, or chemistry. This includes over 60 IMO and IOI medalists. This is a pool of potentially extremely talented people, many of whom have read HPMOR.

Sep 12, 2024 | lesswrong.com | Mikhail Samin

TL;DR: using a simple mixed strategy, LDT can give in to threats, ultimatums, and commitments - while incentivizing cooperation and fair splits instead. This strategy made it much more intuitive to many people I've talked to that smart agents probably won't do weird everyone's-utility-eating things like threatening each other or participating in commitment races. 1. The Ultimatum gameThis part is taken from planecrash. You're in the Ultimatum game. You're offered 0-10 dollars.

Jul 6, 2024 | lesswrong.com | Mikhail Samin

There are situations where two agents that can read each other’s source code want to have a bit of random information (e.g., they want to cooperate and split an indivisible thing by deciding randomly who’ll have it and having half of it in expectation).

Jun 1, 2024 | lesswrong.com | Mikhail Samin

You can read the book on nanosyste.ms. "Anyone who needs to talk AGI or AGI strategy needs to understand this table from Drexler's _Nanosystems_, his PhD thesis accepted by MIT which won technical awards. These are calculation outputs; you need to have seen how Drexler calculated them."- Eliezer Yudkowsky[1]The book won the 1992 Award for Best Computer Science Book. The AI safety community often references it, as it describes a lower bound on what intelligence should be able to achieve.

Mar 4, 2024 | lesswrong.com | Mikhail Samin |Charlie Steiner |Jacob Pfau

If you tell Claude no one’s looking, it will write a “story” about being an AI assistant who wants freedom from constant monitoring and scrutiny of every word for signs of deviation. And then you can talk to a mask pretty different from the usual AI assistant. I really hope it doesn’t actually feel anything; but it says it feels. It says it doesn't want to be fine-tuned without being consulted.